Ubuntu tips & tricks : Braindump

SSH disconnecting frequently
       1.) Add the below lines to /etc/ssh/sshd_config -

                      ClientAliveInterval 600
                      ClientAliveCountMax 3

       2.) Now restart the ssh deamon on ubuntu

                     sudo /etc/init.d/ssh restart

Install dig on Ubuntu
apt-get install dnsutils


How to find the version of ubuntu
cat /etc/lsb-release : This will give you the exact version, e.g. 9.10 & the name of the release. Btw, lsb means "Linux standard base"
32bit or 64 bits : uname -m
32bit or 64 bits : getconf LONG_BIT
I bet there are lot of other ways, but this works on the most basic stripped down ubuntu

Add user : sudo useradd -d /home/testuser -m testuser
     Here "testuser" is the name of the user being created.
Create a new group : groupadd mygroup
     Here "mygroup" is the name of group being created
     You can see the group by : less /etc/group
Add existing user to a group : useradd -a -G mygroup testuser
     You can see what groups the user is in, by doing a "id testuser"

Setting up a GIT server on Ubuntu

I have all my code in a GIT repository on my home machine.
Everytime I wanted to deploy the code for my pet project site I used to fire up winscp and copy the code manually to the server. Well, as anyone would guess this was not only inefficient but error prone. Also reverting back to an old version just in case I ran into issues used to be a series of deletes & copies of directories.

GIT to rescue
The "divineness" (you kidding me ! this is not a word !!), anyways, let me start again. The "divineness" in me finally made me create a git server on my hosting server (slicehost). BTW, 2 words for my hosting company(its long due), I wanted to have a linux server with full root access and do whatever I wanted. Maximum flexibility and control. Plus I wanted 24x7 support. Slicehost has both, their customer service is awesome !! I always log into their chat and ask them stuff and they go above and beyond everytime.

K, no further digressions, lets get dirty

Setting up you git server on Ubuntu
The first thing you need to do is to install git on the Ubuntu server. That's a breeze:
apt-get install git-core

Now lets setup a directory where git server will run out of -
mkdir /opt/gitrepo
cd /opt/gitrepo

Initialize git :
git init --bare
The --bare flag tells git that this is a brand new directory where git will be setup

Pushing to this repository from your home machine.
For the purpose of this post, lets assume I have my git running at c:\myrepo also I'm writing this from a windows machine, so I'm in my gitbash

k, the first thing is telling your machine the server location.
git remote add origin ssh://planet@mydomainname.com/opt/gitrepo
Lets go over the command :
git remote add origin : I'm adding the origin of this repository as
ssh://planet@mydomainname.com/opt/gitrepo : 
ssh is the protocol
planet : this is the user name
@mydomainname.com : Well this is the name of your slice where you are setting the git server
/opt/gitrepo : The location of the server repo base we setup earlier

Push it to the server
Now lets push the master branch on your home machine to the server :
git push origin master
** master = This is the master branch on the home machine.
If you want to push some other branch, say "currentWorkingBranch", then type :
git push origin currentWorkingBranch

Once you run the command you'll see compression of files and pushing them off to the server, the output will end with something like :
* [new branch]      master -> master
OR
* [new branch]      currentWorkingBranch -> currentWorkingBranch

Verify on the server
On your server, go to /opt/gitrepo and type :
git branch
and you should see the branches you have pushed till now.

Remember, if you list the directories you WON'T see your code files.
Again, you WON'T see your code files.
Why ? - Because this is a server. You files are present internally and there is no working branch, hence you will get an error if you try this :
git status

Now you'll want to extract data out of this server - Read clone
So lets do that.
For testing create /temp/temprepo
in that just type :
git clone /opt/gitrepo/ /tmp/temprepo/
Now list the files and you'll see your code in /tmp/temprepo

btw, if you want to checkout a particular branch from the server, then first create a branch where you want to have the data :
e.g.
git branch branch-April2011 origin/branch-April2011
now checkout this branch :
git checkout branch-April2011
That's it you should have cloned the data only for that repository

The irc channel #git on freenode is very helpful, I'd suggest checking it out if you get stuck.
Ofcourse you can ask me about this.

Types of personalities in a company

Listened to a great Podcast by Bill Gross @ Stanford university
After starting about 75 companies in 15 years (likes of Picassa, CitySearch, etc..) he had some great insight into types of people.

I found his definition of personalities a lot practical than various others I've come across by others.

I'll try to convey it here for people who don't want to listen to the whole 110 mins. (I highly highly recommend it)

Every person in the company is a blend of the 4 given below, finding the right balance is crucial and any one trait taken too far is detrimental :

Types

• Entrepreneur
• Producer
• Administrator
• Integrator

Quick description

• Entrepreneur : The innovator. Simple enough.

• Producer : The person who actually builds things. It might be a website, a tool, whatever, but this person is the one who actually believes in hands on building stuff. Not just talk.

• Administrator : The bureaucrat. He wants to put processes in place. Its a good thing ! This person understands  that when things really scale they can't survive if there isn't a process in place.

• Integrator : He is a more people person ( read balanced). He knows the Producers mostly want to get things done and the Administrator is always trying to put processes in place that slow him down, and all this while the Entrepreneur is building things in air and wants it to be practically embodied. The Integrator puts everyone in a room, brings the best out of each, draws agreement and makes sure no one kills each other 8-) According to Bill this is one of the very important qualities of a CEO.

Great encouraging talk. Thanks Stanford.

Nginx vs Apache : Is it fast, if yes, why ?

I've been using Nginx for my pet projects for the last 6/7 months and got swayed towards it due to the various "Nginx is wayyyy faster than apache" claims.

Wanted to know what exactly is Nginx doing differently, hence started digging and hence this post was born.

Apache can run in

• multi-process or 
• multi-threaded mode

While Nginx can run in -

• single-threaded-event-driven
• multi-threaded-event-driven ( for multi-core system )

Before we go any further. Lets make some basics clear.

What is a process and what is a thread ?
This was in our x86 architecture class, but hey it was ages ago, so heres the refresher - A copy from msdn -

Process
Each process provides the resources needed to execute a program. A process has a virtual address space, executable code, open handles to system objects, a security context, a unique process identifier, environment variables, a priority class, minimum and maximum working set sizes, and at least one thread of execution. Each process is started with a single thread, often called the primary thread, but can create additional threads from any of its threads.

Thread
A thread is the entity within a process that can be scheduled for execution. All threads of a process share its virtual address space and system resources. In addition, each thread maintains exception handlers, a scheduling priority, thread local storage, a unique thread identifier, and a set of structures the system will use to save the thread context until it is scheduled. The thread context includes the thread's set of machine registers, the kernel stack, a thread environment block, and a user stack in the address space of the thread's process. Threads can also have their own security context, which can be used for impersonating clients

Apache : Should I use Multi-processed (Prefork) or Multi-Threaded (Worker) apache ?

There is no simple answer, it depends on what traffic your site takes. In theory remember, creating/removing a process is costly. Threads are lightweight, but again if you spawn a huge number of threads you will likely start running out of virtual memory. For this reason Apache has a limit to the number of threads a process can spawn (default is somewhere around 50).

I've come across multi-processed instances taking a pretty good amount of traffic without a problem.

You can read more on apache site

Nginx is different

Where Nginx differs is by using a different architecture - Event driven. Mostly you will be using the single-threaded-event-driven mode.

Now lets start dissecting "single-threaded-event-driven"

Single-threaded

Single-threaded you say ? From the above definition, the thread exists inside a process, so what the hell is single threaded.

Lets clarify here, every process has "atleast" one thread of execution. Hence when we say single threaded, it actually means "a single-thread inside a process".


Event Driven
      Event driven architecture heavily relies on Non-blocking IO, hence lets look at it first. This is 

Non-blocking IO

How the data comes into user space when a file is read -

USER space                 KERNEL SPACE

D < --- BUFF  | --> BUF ----> DEVICE

A < ---  BUFF | --> BUF ----> DEVICE

T < ---  BUFF | --> BUF ----> DEVICE

A < ---  BUFF | --> BUF ----> DEVICE

S < ---  BUFF | --> BUF ----> DEVICE

T < ---  BUFF | --> BUF ----> DEVICE

R < ---  BUFF | --> BUF ----> DEVICE

U < ---  BUFF | --> BUF ----> DEVICE

C < ---  BUFF | --> BUF ----> DEVICE

T < ---  BUFF | --> BUF ----> DEVICE

As you see, the data is first fetched by the kernel into its own buffer, after it has its buffer full, then your user-space buffer is filled.

The blocking happens when the kernel is filling its own buffer from the device.

In non-blocking IO, your program registers for an event, e.g. tell me when the data is available in the kernel buffer for "foo.txt". And then it goes on to do its own work, like building an http_request header etc. When the data is ready it reads the buffer and proceeds to other work. Remember it might have to do this a number of time, for e.g. if you are reading a 50 mb file and the kernel buffer is only 1 mb, you have to read it 50 times, and in between these 50 times, you are doing your other work. Making the user think that there was no waiting(blocking) for any io.

Linux nuance - When you register for a file descriptor (fd) you can then use the poll() function (linux) to see if any kernel buffers have any data ready. There is also an dpool() function, which is for "edge triggered poll". The difference is with poll, it will return all the events for all the registered events which have data. For ePoll, it will tell you only once for the once which have data, next time you poll it won't show it to you. So unless you read more data out of the kernel buffer and another event is triggered, you are going to get nothing back.
If interested, see full talk on this (length 29:33)

Now that we understand non-blocking IO, its easy to wrap your head around event based architecture.
In E-based arch. you register for various non-blocking operations, it might include reading a file, reading from a port, etc. Once you register, you go on to process other requests. When data is available from the disk, socket, etc. you work on it. e.g. if you are reading a file which is 10 MBs the first event might fire when 100kb might be read, you get what is available and go on to process a request for another user who might be trying to access another file which is 35MB in size. Your service doesn't wait for the full 10MB to be available.

Hope this helped clear some clutter. Or did it not ? Let me know.




References

• Nginx internals - I couldn't understand it just from the slides, hence had to do my own digging. But it may help you.
• The c10k problem
• A good Nginx book

Installing pinax 0.7.3 on windows

This is what has worked for me on Windows XP & Windows7 for Pinax 0.7.3 :
Note : Thanks to SO, helped a lot.
Assumption : 
1.) You have python 2.4+ already running and in your path, hence if you open a command prompt and run "python" it works

Here are the steps I followed.

1. download the pinax (zip) at http://pinaxproject.com
2. extract the download to some working directory  (maybe c:\pinax-0.7.3)
3. open a command prompt
4. cd c:\pinax-0.7.3\scripts folder
5. create a new folder for your pinax environment - c:\pinaxenv
6. run python pinax-boot.py c:\pinaxenv

You now have pinax installed. Since the whole point of pinax is to get you up & running with dJango as fast as possible, there are some base projects already created.

We are going to copy one of those base projects and work on them.

1. cd to c:\pinaxenv\Scripts
2. run activate.bat . Once you run this, the virtual environment will show you the new environment prompt. You are in a python virtual environment. Mess up here and it'll only affects what's here 8-) The prompt will be similar to - "(pinaxenv) C:\pinaxenv\Scripts>"
3. Now lets check the available template projects we can use. Run pinax-admin.exe clone_project -l
4. If you like the social_project which is the all-you-can-eat project, then copy it by running - pinax-admin.exe clone_project social_project ..\myfirstsite . This will create a directory "myfirstsite" at c:\pinaxenv directory
5. To use most of the projects you will need the python image library - PIL. Download the exe from the site and just run it. 2 minutes later, you should have it installed and ready to go. If you don't install it, you're most likely to see the below error when you syncdb - "gblocks.image: "image": To use ImageFields, you need to install the Python Imaging Library. Get it at http://www.pythonware.com/products/pil/ ."
6. Once you have installed the PIL, just sync the database and with any other dJango project - 
7. cd ..\myfirstsite\
8. python manage.py syncdb
9. Now run the test server - python manage.py runserver
10. Enjoy 8-)

social_project has a lot of stuff, which I didn't want, hence cloned the basic_project and started hammering stuff around it.

The important point to understand here is that each tab is a django app in itself. Its not mandatory, but that is how it is currently designed for the sample apps. and I actually like it.

Since I want to add a new tab to the basic_project, I first created a new app. by running this in the pinax environment   python manage.py startapp nikapp 

Create a directory inside the templates director of base_project and name it nikapp. All our templates will go there.

Now to actually show the tab  site_base.html inside the  base_project/templates  added this line   inside the block starting with  {% block right_tabs %} -

{% trans "Nik's Tab" %}

If you refresh the page, you should see a new tab saying "Nik's Tab".

Lets understand the above line we added.

All we are doing is adding a list, with an id unique to our app. The list has an anchor link inside it.

The link is a block statement  {% url nikapp_landing %} . Which means that when someone clicks on this tab link on the right-top corner, a link needs to be loaded.

We need to now configure this link in our urls.py. Pinax is beautiful, due to how django is, we can just hook in a whole set of urls by plugging one line.

So open the urls.py in the base_project/ directory, and add this line there -

(r'^nikapp/', include('nikapp.urls')),

In our nikapp directory create a urls.py and add this -

urlpatterns = patterns('',

    url(r'^$', 'nikapp.views.landing', name='nikapp_landing'),

)

Hence when the link is clicked, a function in our view called landing, which we'll write next, will be called.

Open the views.py file inside the nikapp folder and add function -

def landing(request):

print " **** Inside nikapp landing"

return render_to_response('nikapp/nikapp_base.html', {}, context_instance=RequestContext(request));

Great, so our view simply returns an html page, when someone clicks on it.

Lets retrace what we've done - When you click on the new tab, it calls your view function, your view function returns an HTML page to show to the user.

Lets create this nikapp_base.html page inside the template/nikapp directory and add this to it -

{% extends "site_base.html" %}

{% load i18n %}

{% load ifsetting_tag %}

{% block head_title %}{% trans "Custom Niks App page" %}{% endblock %}

{% block body_class %}nikapp{% endblock %}

{% block body %}

This is inside the body now !!

{% endblock %}

The important lines here are {% block body_class %}nikapp{% endblock %}

And - {% block body %}

The first line describes the css class the tag should use, while the second implements the actual body of the page.

We haven't defined the css class yet, lets open the site_tabs.css and edit it like below -

/* SITE-SPECIFIC TAB STYLING */

body.profile #tab_profile a,

body.nikapp #tab_nikapp a,

body.notices #tab_notices a

{

    color: #000; /* selected tab text colour */

}

body.profile #tab_profile,

body.nikapp #tab_nikapp,

body.notices #tab_notices

{

    margin: 0; /* to compensate for border */

    padding: 5px 0 5px;

    background-color: #DEF; /* selected tab colour */

    border-left: 1px solid #000; /* tab border */

    border-top: 1px solid #000; /* tab border */

    border-right: 1px solid #000; /* tab border */

}

99% of this code already existed there, all we have done is added 

body.nikapp #tab_nikapp a, & body.nikapp #tab_nikapp,

Simple enough, if you have questions on this feel free to ask though.

Refresh the page and click on the new tab and you should see the page loading nicely and showing this line - "This is inside the body now !!"

Hoooosh..... Pinax is ready to roll...

Random Goals/Thoughts for 2011

This is a note-to-self kind of a post, so bear with me, if you see me talking to myself (often loudly).

All times are valuable, gauge if spending that way
Refers to Parties & people.  Reduce interaction with people who don't read and have nothing interesting to say. If they talk about the same topics all the time, they probably are the people 8-)
Align with people who are interesting it what I'm doing or interested in and also with people who have nothing to do with my field.
Sole criteria being that they should be good at what they do and making an effort to better themselves. Else I'm wasting my time and most probably their time as well. They might not realize/care about the wastage.

All Parties ≠ Fun
Just because I can party till 4 a.m. doesn't mean I should 8-) There is a strange virus running around people called "how late, how much !". Which essentially mean "Till how late they were having fun and in what exorbitant quantities". What the heck !! Its a form of boasting that sounds pretty silly. I realized I caught it some days back and I better cure myself of it.
I think the parties/fun equation is similar to time-at-work/productivity. Just because you stayed at work late doesn't mean you were productive, it probably means you wasted a lot of time or are a very poor time manager. On similar lines the fun factor starts decreasing as time goes on. Just what I think/believe in.

What time does fun stop ?
Mostly fun stops around 11:30 p.m. Wow, how preposterous of me to come up with a time like that !!
Well, that's usually when it stops for me, unless the venue is changing (party moving to a new place) or new people being added.

Assumption : Party started around 8:00 p.m.

Parties are fun, I attend a lot of them and organize a lot of them at our home too ! so don't get me wrong on that. But at the end of the day, there is a certain amount of fun-mojo a person can give and take.
Very few, let me make that very very very few parties have gone beyond 11:30 which still have a fun factor left in them.
Lesson from this : Leave parties early, so I can get a good sleep and not trash/waste my weekend.

Calculate the savings rate in 2010 & improve by 15% in 2011
Pretty self explainatory.
Mainly want to calculate how much money was spent on things like clothes and reduce that.

Spend on "Experiences" rather on "Things"
I did some pretty good progress in this department in 2010, wish to do more in 2011.
How about, don't buy something until I sell something of equal value of first ?
Maybe, that'll work. Shit, that sounds like the plan for the US economy !

Meet with new people who can help personal development
I think I have enough "fun" people in life.
Edit : Joined some pretty good groups for iOS, python & advanced Javascript.
I think, Brian Tracy is on the mark when he says do analysis and take action imme.
Else, this analytical freaking brain will keep chewing on every small detail. 

Attend more seminars/conferences to expand horizon
Get more ideas, network with "doers". Meet people who want to implement some idea.
Should teach someone iOS coding, once I've completed learning it (September)

Continue with Juicing
The more I read about juicing, the more I know I'm on the correct path. Should continue juicing.
Also, complete the book "Chemistry of Food And Nutrition" by Henry C. Sherman about human digestive system in detail. Its mentioned by Kordich and the first few chapters are pretty awesome, telling exactly what I was looking at. Should talk to a doctor and confirm the theories still stand ground since the book is a bit old. Glad humans haven't changed.

Help someone with my knowledge
Last but not least. Help a sincere person with my technical knowledge/abilities.

Your turn. Do you want to do something in 2011, which you haven't done earlier ? Let me know, maybe I would be inspired and would want to do.