Content

Saturday, October 27, 2018

How to update git credentials after changing GitHub.com password?

Once you have changed your password in GitHub.com do the following in the git command line

You would notice that existing git pushing don't work and you would get a fatal error.
The command to execute is
 git config --global credential.helper osxkeychain 

The flow would be as follows, once you get the error issue this command and then try pushing to GitHub.com, it would now prompt the credentials again to be updated.

$ git push origin master remote: 
Invalid username or password. fatal: Authentication failed for 'https://github.com/[yourhandle]/[yourrepository].git/' 

$ git config --global credential.helper osxkeychain 
$ git push origin master 
Username for 'https://github.com': [yourhandle] 
Password for 'https://[yourhandle]@github.com':

Sunday, September 30, 2018

How to configure GitHub for first time use?

Github.com is an online enhanced version of Git.

Git is locally installed in each of your work stations. This is then used as a client connecting to the server version of git which is GitHub.com

In a way what you are doing is like an offline web application, you do all the work in your local computer and when you are done with the work, you sync up with the server. There is no need for Internet connection etc.

Git is usually installed in your laptop either by installed XCode in Mac


Go GitHub.com and register an account for yourself. It can range from a free account, a private account or a company account. The various description are given in GitHub.com

Once done on your laptop do the following

## This is required for making sure that all your commits done on a repository have the correct name
git config --global user.name "<Your GitHub.com Handle>"
git config --global user.email "<Your Email Address used for registering to GitHub.com"


## Create a new repository called GitHubHelloWorld in GitHub Online
## Create a Directory on the local computer
mkdir GitHubHelloWorld

move to the directory and give the following command
git init

# create a new file
readme.txt and edit the text

# add the file to git
git add readme.txt

# commit the file
git commit -m "Add readme.txt"

# Add a remote origin in preparation for pushing to GitHub.com.
# You need to add this only once
git remote add origin https://github.com/<youruseraccount>/GitHubHelloWorld.git

# Push the changes to GitHub.com under master. More on this later on the branching strategy
git push origin master


# Some useful commands
# Find all the files which are changed in Local which is not synced up with server
git status

# Short cut to add all files to git for the changes
git add --all

# Commit all the files
git commit -am "Initial Setup of Maven Project"


# If you don't want certain files to be in the repository, you need to add a .ignore file. This file would have the extensions or the file pattern you need to exclude from source control



Saturday, November 11, 2017

How to read an Apache Storm 1.0.2 Kafka Spout Zookeeper offset?

As with any streaming systems, the consumers namely the Kafka Spout need to keep track up to which message identifier has been read in Kafka topic. The reason this needs to be persisted is if we do need to restart the topology and the spout restarts, it needs to know from where it needs to start reading from else it would start from the beginning. In order to prevent this from happening Storm Kafka spout allows once to persist the offset in Zookeeper and would automatically read it when the Topology restarts.

In order to find this information,  we need to login to the Zookeeper Command Line Shell.
cd /usr/local/zookeeper/zookeeper-3.4.9

bin/zkCli.sh -server zookeeper1:2181

Check the topic and its partition for the consumer, your need to type in your topicname and if you have more than 1 partition make sure you do this to every partition

get /consumers/yourcompany/yourtopicname/partition_0

Sample response

{"topology":{"id":"YourTopologyInstanceId-1-1497152721","name":"YourTopologyName"},"offset":3673,"partition":0,"broker":{"host":"81387110753b","port":9092},"topic":"yourTopicName"}

The offset:3673 says up to which offset the Kafka spout has read from this topic's partition 0.


Configure how long a message is kept in Kafka 0.10. 1.0 topic?

Sometime, due to disk size issues or just the fact that we don't want old messages in Kafka, one might need to clear these messages in Kafka topic automatically.

Please note, as of the current writing Kafka is not clever enough to clear messages once they are read by any one consumer. This needs to be done manually by the User. So do make sure that you have a setting that fits your purpose. i.e your consumers should be able to read it before the message is deleted from Kafka. Once a message is deleted from Kafka there is no way to get it back.

The current default for Kafka messages in a Topic is 7 Days.

Here we are changing this to 1 day.

cd /usr/local/kafka/kafka_2.11-0.10.1.0/bin

./kafka-topics.sh --zookeeper zookeeper1:2181 --alter --topic yourtopicname --config retention.ms=86400000

How to clear all messages in Kafka 0.10.1.0 topic without deleting the topic?

Sometime, the messages in Kafka topic would be overwhelming and we need a quick way to clear these messages without deleting the topic.

Note deleting the topic is an option that should be used with caution in Production.

Kafka topics by default have a concept of retention, i.e how long a message in a topic needs to be persisted. By default this setting is 7 days. We are going to make use of this to clear the messages in the topic.

We are going to set the retention to 1 second and then bring it back to 7 days. Login to the Kafka directory and go to the folder location where Kafka is installed.


cd /usr/local/kafka/kafka_2.11-0.10.1.0/bin

./kafka-topics.sh --zookeeper zookeeper1:2181 --alter --topic yourtopicname --config retention.ms=1000


Wait for some time so that the messages are cleared. Check if there are any messages by reading from the topic from the beginning

 bin/kafka-console-consumer.sh --bootstrap-server localhost:9092 --topic yourtopicname --from-beginning 

Make sure that you reset the timing to the original 7 days

Important Please do not forget to do this step, else all your messages in Kafka would be deleted after 1 second delay.

cd /usr/local/kafka/kafka_2.11-0.10.1.0/bin

./kafka-topics.sh --zookeeper zookeeper1:2181 --alter --topic yourtopicname --config retention.ms=604800000

Sunday, December 18, 2016

How to work with Github.com Fork Pattern?

Login to the github account. Make sure that you have access to the repository which you want to fork.

https://github.com/githubacountoffork/repositorytofork

Hit the Fork button. Once you click on the Fork button the repository would be forked to your github account.
https://github.com/yourgithubaccount/repositoryoffork

This fork would be the origin for you to push changes to the repository that you forked. Basically you are making changes to your local fork and then giving pull request to the original source where the actual repository is.

Clone the forked master repository from the github to your local development machine.
git clone https://github.com/yourgithubaccount/repositoryoffork


Add the remote upstream to say where to get the remote repository
git remote add upstream https://github.com/githubacountoffork/repositorytofork

Verify the new remote named 'upstream'. You should see both 'origin' and 'remote' pointing back to the server link.
git remote -v


As a best practice do not work on the local master, create a  branch for every story or task
git branch branch-name


Check the branch that you have created.
The * on the branch name indicates the current active branch.
git branch

On of the greatest feature of git from other repository is that you don't have multiple folders for different release of repositories like mainline, dev, release 1 etc. You just switch between branches using the same physical repository.
The downside is that you can work on only one branch at a time. We need to ensure that we are in the right branch all the time. To switch to any given branch use the following command
git checkout branch-name

Work on the files, using IDE of your choice, to check which files are modified issue
git status

When you are done doing the changes, add those files to git and issue a commit before working on new files for committing.
git add .

To selectively add the files, go to the individual folders run the add command  using the file name or folder name.
Do a Git status and check the message about 'pending commit'. The add command has only 'staged' the changes. The changes are not committed yet.
git status

The next step is to commit the change using the following changes.
Commit will take all the 'staged' files and commit to your local branch.
git commit -m 'commit message detailing the changes.'


Check status and make sure changes are committed.
git status

Push these changes to your fork, if the branch name is not present, this branch is automatically created
git push origin 'branch-name'

Now log on the github.com website.
From the drop down select the branch name that you just 'pushed'
Create a pull request targeted to the Origin repository
This will send a pull request to the reviewer who can review and merge the changes.
After the pull request is merged to the main master, we have the option to Delete a branch. Remember we have a local copy and a remote copy. So we have to delete both. Sometimes the remote fork branch copy can be deleted by the Pull Request taker. In this case there is no need to delete the remote branch on the fork.

To delete the local branch, move to the master branch before deleting your local branch
git checkout master

Force delete the local branch
git branch -D branch-name

If the branch is not deleted by the Pull Request (PR) on the Remote Branch
git push origin branch-name

Once manual step with this approach is that you need to synchronize the remote repository to your fork. This is not done automatically as sometimes people just want to work on that version of the fork only. This is not enforced by Github.com

Get all changes from remote
IMPORTANT: make sure that you have the correct working branch before you issue this command else your would replace your other branches and conflicts can occur.
git branch

Pull changes from the remote upstream branch
git pull upstream master

Push this changes to your local fork branch
git push origin master


How to switch default java version namely Java 7 to Java 8 in Mac OS X?

Once in a while Java versions on your developer environment needs to be upgraded and the new version needs to be the default across the different software that you would be using for the product or solution.


Before changing the default, find the current default in the system by typing

java -version

This should give something like 
java version "1.7.0_79"
Java(TM) SE Runtime Environment (build 1.7.0_79-b15)
Java HotSpot(TM) 64-Bit Server VM (build 24.79-b02, mixed mode)

Now check the other versions installed on the system

/usr/libexec/java_home -V

This should give something like
Matching Java Virtual Machines (2):
    1.8.0_45, x86_64: "Java SE 8" /Library/Java/JavaVirtualMachines/jdk1.8.0_45.jdk/Contents/Home
    1.7.0_79, x86_64: "Java SE 7" /Library/Java/JavaVirtualMachines/jdk1.7.0_79.jdk/Contents/Home

Now change the version to Java 8

export JAVA_HOME='/usr/libexec/java_home -v 1.8.0_45'

Verify if this is changed by giving the same command again

java -version

This should now show the changed default
java version "1.8.0_45"
Java(TM) SE Runtime Environment (build 1.8.0_45-b14)
Java HotSpot(TM) 64-Bit Server VM (build 25.45-b02, mixed mode)