Look who is talking about responsibility

While traveling around each day I bounce in to different people. Today while taking the taxi to the Changi I met up with an interesting taxi driver. He was a tall, Chinese, middle aged man. As soon as I got into the Taxi I said "I want to go to Changi" and then he asked "what was the terminal?" I replied, “I think Terminal 1...but let me check the ticket and confirm", so while I was digging my bag to get the ticket out. He told me “Tell me the air line la, I might be able to tell the terminal", as soon I had found the ticket and told him, “its Emirates Airlines and Terminal 1”. Then this guy started to explain to me about the airlines and the terminal relationships. I never new anything like that, although I had been to Changi a few times.

Then he explained to me why he asked for the airline, I was utterly surprised by this explanation. He told me even if the passenger has mistakenly told him a wrong Terminal number; it was the RESPONSIBILITY as the taxi driver to use his knowledge in the best possible way to help the passenger to get to the right terminal. Now I thought for a moment and was thinking was it “an extended self imposed responsibility” that this guy was talking about. Then I was trying to map to my job. If customer gives me a wrong requirement which will be a hindrance to him and he cannot foresee, isn’t it my professional responsibility to go that extra mile to make the relevant suggestion with the best of my knowledge?, so for my profession I thought the answer was an yes, This led me to analyze my own thinking process. I was thinking at least for a few minutes why did consider this was unnecessary RESPONSIBILITY of a taxi driver. Then I realized that the mistake that I made was that I did not consider that the taxi driving was a professional occupation, within myself I felt ashamed, after seeing the infamous three wheel drivers around this place , I had grown up never thinking that this was a profession which has a set of ethics and RESPONSIBILITIES.

This was the point at which I realized why the Singapore Taxi drivers stood out. A level of service I had never found in any of the other countries I had been to, it’s not about luxury but a service known for its highest level of trust and discipline.

Then he inquired from me whether if I was reading the paper while I was here. I said I didn’t get to read any local Singapore news papers and was only checking the international news websites last few days. He then related to me the story of a taxi driver who “threw” out two ladies from the taxi in the middle of the night in a jungle. This sounded weird, in Singapore standards by a jungle; I think he might probably be refereeing to a patch of road with trees on both sides of it. Still he was really angry with his fellow taxi driver to have done this and bought discredit to his profession. He explained to me that the taxi driver, when in a position where he is unable to find the place to drop the passenger, he should have called a friend to get the location information or at least should have dropped the two ladies at a safer location at a highway or a mall.

He was so proud of his company, when he said the taxi company sacked the driver and his license was removed. He then told me if this was not done people will start to assume “he can do, I can do, and we can do” and it will spread like a virus and become an accepted practice”!!! Then he hit the nail on the head when he told a Chinese proverb and translated it to English which was something similar to “Kill one to stop hundred”. I usually hate to hear proverbs talking about killing but in this context I should confess that I liked it.

MySQL Development Best Practices

I strongly believe that MySQL programming best practices need to be known and followed by each developer, since practicing query optimization from day one make everyone's life much more easier. So I present here a very simple MySQL Development check list.


Check List for General Best Practices

1. Checking for indexing

1.1 All columns in the 'WHERE' clauses indexed?

• [Eg. select * from student where StudentId = '7986'; then column StudentId need to be indexed]
  

1.2 All columns in the 'ORDER BY' indexed?

• [Eg. select * from student ORDER BY dob; then column dob need to be indexed]

1.3 All columns in the 'GROUP BY' indexed?

• [Eg. select * from student GROUP BY Class; then column Class need to be indexed]
  

1.4 All columns used for joins indexed?

• [Eg. select country.Name, city.Name where country.Id = city.CountryId; both country.Id and City.CountryId are better to be indexed]
  

2. Checking for over indexing

2.1 No redundant indexes?

• when having composite indexes remember that the left most indexes can serve as indexing and does not require separate indexes.

• [Eg. select * from student where house = 'YELLOWS' and class = '3E';

• if student table has the following indexes a. index(house,class) b. index(house)The index(house) is a redundant index hence not required and better be removed.
  

2.2 No indexes that has been created and never used within the 'WHERE'or ORDER BY or GROUP BY or for joins?

2.3 No simple indexes created on column which does not have unique values?

• [Eg. A simple index on a 'sex' column with a data set with a distribution of 45% Males and 55% Females will not be usually used for result pruning]

3. Check for data types

3.1 Has the appropriate shortest data type been chosen for each column?

• [Eg. If a column can be defined as TINYINT do not have it as BIGINT]
• [Eg. If a column can be defined as varchar(10) do not define it as varchar(255)]
• [Eg. If a column can be defined as enum do not define it either as varchar, char or int]

3.2 Has the integer columns not taking minus values defined as 'unsigned'?

3.3 Has the columns which could never have NULL values explicitly defined as NOT NULL?

4. INNODB Specific Checks

4.1 No char columns in INNODB tables?

• [ Do not use char columns in INNODB always have VARCHAR]

4.2 Does All the INNODB tables have primary key?

4.3 Are the columns in the Primary key of the minimal possible length?

4.4 Has all the statements like, select count(*) from innodbTable; statements eliminated?

• [Specially do not use on critical tables with a huge dataset. As an alternative to the select count(*) use summary tables]
  

* NOTE: I had avoided complicated practices and included only the most simple practices.

panorama pics

I had become sort of addicted to taking panorama pics while I travel around. If I am to confess the reason why I bought this camera [Kodak M853] its actually because of three things(mainly),
1. Suits my limited budget
2. Has capability for night photography and shake free.
3. The talent of the sales person by showing me the panorama mode.

Now I feel that I had slowly staring to master it and here are some panorama pics taken in Singapore.

Jealousy - Staring at Aish

It has been long time since I wrote any blogs. But today I was tempted to write about jealousy. A topic that could generate a lot of controversy. So instead of doing any writing I thought that I would let this picture do the talking.

Who said a picture is worth a thousand words? Its absolutely lot more than that...lol

*** Pic with make up

Using Perl Script for memory calculation of NDB Cluster in Fedora 9

Step 1: Check that you have perl

get the perl DBI module installed by using,

yum install perl-DBD-MySQL

step 2: cd into $MYSQL_HOME(or mysql base directory usually /usr/local/mysql and try

./bin/ndb_size.pl --database=myDb --user=myUser --socket=/tmp/mysql.sock --password=password

If this fails with the error like Can't use string ("9/16") as a HASH ref while "strict refs" ..... then you need to get fix the bug by replacing the following peice of code,

current code, in release ..it starts from line number 920


foreach my $i(@show_indexes)
{
$indexes{${%$i}{Key_name}}= {
type=>${%$i}{Index_type},
unique=>!${%$i}{Non_unique},
comment=>${%$i}{Comment},
} if !defined($indexes{${%$i}{Key_name}});
$indexes{${%$i}{Key_name}}{columns}[${%$i}{Seq_in_index}-1]=
${%$i}{Column_name};
}


need to be replaced as follows,

foreach my $i(@show_indexes)
{
$indexes{$i->{Key_name}}= {
type=>$i->{Index_type},
unique=>$i->{Non_unique},
comment=>$i->{Comment},
} if !defined($indexes{$i->{Key_name}});

$indexes{$i->{Key_name}}{columns}[$i->{Seq_in_index}-1]=
$i->{Column_name};
}


then executing,

./bin/ndb_size.pl --database=myDb --user=myUser --socket=/tmp/mysql.sock --password=password

should get the output for you.

Looking to optimize "Group by" in MySQL?

Few months back I was asked for the reason behind the recommendation by the consultants from MySQL. When the system was performing poorly the consultants had walked in and simply requested them to add the "order by NULL" clause to the queries. To the joy of the client and amazement of the tecnical team at the client location, this simple fix had done the trick and suddenly there was a marked improvement in performance! This had left the technical team confused, as they started to wonder why and how this happened. WhenI heard this I was confused ( as always ;) ), as I could not think of any possible logical explanation for this behaviour.

Last week while I was reading the planet MySQL I bumbed into an article talking about order by NULL and I think I had found the possible reason. Any how it is not a statement, which would optimize all queries , instead it would optimize all the statements with a group by clause.

To make things simple lets take small example,

EXPLAIN SELECT CountryCode, COUNT(*) FROM City GROUP BY CountryCode \G

This gives the following output,

*************************** 1. row ***************************
id: 1
select_type: SIMPLE
table: City
type: ALL
possible_keys: NULL
key: NULL
key_len: NULL
ref: NULL
rows: 4079
Extra: Using temporary; Using filesort

The extra column indicates that both filesort and temporary is being used, which is an indication of poor performance.

Alternatively when its applied with a Order by NULL as shown below,

EXPLAIN SELECT CountryCode, COUNT(*) FROM City GROUP BY CountryCode order by NULL \G

gives an ouput,

*************************** 1. row ***************************
id: 1
select_type: SIMPLE
table: City
type: ALL
possible_keys: NULL
key: NULL
key_len: NULL
ref: NULL
rows: 4079
Extra: Using temporary

Which indicates a bit of an improvement as it does not any longer have Using filesort.

This addition of a order by NULL could give a performance increament by many folds at times.

The next most important question is how does this happen?

MySQL by default when having a group by statement would also order by the same column, which would require an additional sorting to take place. Any how when the order by NULL is included, it does not do a sort and thereby give a performance improvement.

Sir, but with love

From my childhood I had been bought up in an atmosphere where the "Sir" was used to refer the teacher's at my school and quiet rightly these people were those for whom we had great admiration for and looked up to as exemplary characters who molded us into what we are today. It’s true that this way of referring to my masters was something that was taught to me or even one could argue that this was a protocol which was forced on to me while I was kid. Whatever the view point the reader may have I still think that while being a kid this should be the way in which the system should continue because as kids its not possible for us to distinguish an ordinary person from a personality to be referred as "Sir" and also else as a kid we would never learn to respect people. Looking from where I presently stand I still call these great teacher’s of mine as “Sir” and with the same or even more amount of admiration and respect for the part they had played in my life. Probably this mentality still is in my mind as I had already done a benchmark of this term “Sir” at a very high level and being a servant in the software industry where the term “Sir” is frowned upon.

The sudden emergence of the practice in the local universities by the “temporary tutors”, who are just a batch or two senior to demand from the rest to refer them as “Sir” or “Madam” seems ironical. It’s an acceptable practice, if it’s a military establishment where you find a culture where people salute the ranks and not the person and as long as I know a university is not a military organization as long as it’s not a military university. I find that these people had been caught up in a “psychological illusion” where they consider them to be lecturers or even in the caliber of professors now it self. The real reason for this being the fact that unfortunately for many and fortunately for some we have a serious scarcity for qualified lecturers and you find that people fresh out from the degree very often doing lectures to degree students. Please note that I am not blaming these fresh graduates for teaching as I myself had been thought by some good teachers who were fresh graduates, but I am actually throwing caution at the people who had got blown out by this new temporary seat they had been assigned to.

The only thing that I can say to them is wait for the turn and at that point people will call you as “Sir” out of respect for the wisdom you posses or the personality you are or for charisma you posses and not to jump the gun to demand or rather in a way beg for respect, remember when you get the respect that you had demanded specially from a university student its just a temporary thing, it does not come from the heart.

No pun indented, no corners spared. Its just some musings from “sayy” so take if something is worth out of what is written or even don’t be hesitative to spit on me(I mean literarily ;) ) by the way of a harsh comment if you really feel that I deserve it for writing this.