Wednesday, November 5, 2008

Friedman's 10 World Flatteners

Thomas L. Friedman's "The World Is Flat" is probably the best non fiction book I have read so far. It gives an amazing insight into 21st century: starting from why things are the way they are, how they will be and what factors will shape them. Here the "flat" is in reference to level playing field where each nation has almost equal opportunity to be a player at a global level. He provides a list of ten forces that flattened the world.

  1. Fall of Berlin Wall (Nov 9, 1989)
  • It was not only blocking our way but was blocking our sight. World was sort of like a frog in a well. Demolition set the frog out of the well.
  • End of communism which in the simplest term is "a great system for making people equally poor".
  • Created a chain reaction of freedom.
  • Simultaneous birth and popularity of IBM PC/Microsoft Windows
2. Birth of Internet Browser Netscape
  • Created a dot com bubble which helped corporation realize the power of internet which ultimately resulted into huge investment in connecting the world through fiber optic network.
  • Dot com bust lead into cheap availability of these fiber optic data transmission network which third world countries could use.
3. Work Flow Software
  • Automated communication between different departments of a corporation. New standards for software to talk to each other (SOAP, XML for Web services etc)
4. Open Sourcing
  • Tell me about that. ;-)
5. Outsourcing
  • Definition: To take a limited function of a company that was being done in-house such as research, call centers and have another company perform the exact same function and then reintegrate their work back to overall operation.
  • Dot com allowed country like India, China and some Eastern European countries to use all the data line practically for free.
  • World could use India's millions of engineering brain power not only in software but in accounting, medical data analysis, customer support
6. Offshoring
  • Definition: When company takes one of its factories say in Canton, Ohio and moves the whole factory offshore to Canton, China.
  • China wanted to attract foreign manufactures not simply to manufacture 1 billion pair of undies for sale in China but to use low-wage Chinese pairs of undies to sell 6 billion pairs of undies to the works at price that were a fraction of what companies in Europe and America.
  • Companies started breaking their business into three parts: part to do in China, part to sell to China and part to buy from China.
  • China flattener has been wrenching for manufacturing workers but a godsend for all consumers. By mid-1990s alone, cheap imports from China have saves U.S consumers $600 billion and saved US manufactures untold billions in cheaper parts for their products. This saving in turn helped Federal Reserve to hold down interest rates longer, giving more Americans to finance home and new business.
7. Supply Chain Management (SCM)
  • Wal-Mart's uber cool efficiency on SCM. Its system is inter connected to such an extent that the moment a customer purchases something, the record goes to all the way to manufactures through corporate headquarter and wholesale dealer without a single intervention from a person.
8. In Sourcing
  • UPS story: When you return a Nikon camera, it does not go to Nikon headquarter all the way. UPS has certified engineers to fix the camera. If they can not fix it, they ship a new camera to customer thus saving on extra shipping that would have been needed otherwise.
  • Def: a new form of collaboration and creating value horizontally.
  • Even small companies can do business globally without opening a shop in rest of the world. UPS acts as your global supply chain manager.
9. In-Forming (easy access to knowledge - Think Google)

10. Influx of Gadgets - Digital, Mobile, Virtual
  • Being able to print boarding pass at home
  • Blogging from anywhere through various devices - even mobile phones.
  • Engines talking to computers, talking to people, talking back to the engines, followed by people talking to people - all done from anywhere to anywhere.
BTW, above is only one third of the book.

Tuesday, September 23, 2008

Lessons Learned from Scalability Testing

Currently I work on JBoss Portal project and recently I have been working on its performance and scalability part. The outcome of my recent effort is published at this blog entry. This was a very prolonged testing which required lots of patience, working with several teams, learning many new different tools and concepts. But now it feels like every bit of it was worth it. Here are the lessons that I learned not in any particular order:

  • Performance and scalability testing is not easy. Besides it being heavily technical I believe it's an art and you mature by experience.
  • Set a goal first otherwise there is no end to this. Be very concrete: I want my application to handle x concurrent users with response time of less than y second/s and it should scale more than z%.
  • If the application/system you are trying to improve the performance of and making more scalable is not developed by you, it's a must that people who developed are aware of scope of your work and helpful. I had good fortune of this being the case. I do not even want to imagine how it would have been otherwise.
  • The best way to find bottlenecks is taking several thread dumps and taking very close look at them. I found it the most revealing and helpful than any profiler (commercial or open source). TDA is a good tool to analyze thread dump. I used three profilers: JBoss Profiler, JProfiler, SAP's Memory Analyzer.
  • Regardless of what people say, open source load testing tools serve most purposes. I found Grinder to be very straightforward, not that hard to configure, and very vibrant community.
  • For managing distributed deployment, use or extend a distributed framework. I used Smartfrog components to manage instances of JBoss AS, start up load generator, start up apache web server and load balancer otherwise I would have had to log into around 15 servers to start/stop/configure these components.
  • Try to optimize most common code execution path. Optimizing code base which crawls but is also used rarely wont help you much.
  • JVM's default garbage collection configurations are pretty good. I have toyed with several configurations and none of those seem to have any significant impact in my use case. My point is that you play with GC configurations when you have fixed other problems.
  • JConsole that comes free with Sun JDK is pretty good tool to observe GC behavior, memory footprint and different logical memory spaces (eden, perm etc) of heap. Use that as a first tool for your profiling.
  • It's a common knowledge that database is always a bottleneck. Hibernate's show_sql switch is a very good friend. Make sure that you do not see queries being executed more than they should. Tuning those bumped up performance quite a bit.
  • The first thing you turn off after you have done basic optimization is logging. It has a big overhead simply because logging requires write to file and disk access.
  • Don't use NFS for testing.
  • If servers are part of cluster, make sure that they all are part of same subnet and secluded from any other network traffic.
  • Find a compromise between loading everything in memory and loading everything on demand (lazy loading). The former may show a good performance in the beginning but crawl once memory fills up and lot of full GC starts happening. The latter will have higher DB utilization. These settings are dependent on nature of your application and nature of how the application will be used. There is no universal setting. Let me know if you find one.
  • This blog is a WIP. I will add more bullet points as I think of them. :-)

Friday, August 1, 2008

Windows XP security hell !!

I visited my uncle's place last weekend. He has a laptop which his two kids use. Obviously, they primarily use it to play online games. Kids use it so much that some of keys such as up arrow, letter p etc are pretty banged up and you have to hit really hard whenever you need to use it. Coming to the point, when he restarted his laptop, all he was able to see was his desktop. No familiar start button, no icons on desktop. It's just a clean wallpaper. They don't know what to do. Here comes the savior, me, who has no idea what's going on. First thought comes to my mind that why would any operating system would allow an anonymous script/virus/malware/adware to modify system configuration to such an extent. I feel even better that I use Fedora where I exactly know what's being installed. People use Windows because it's easier for people. Tell that to my uncle and kids who can not do anything with their laptop. Here are other things that did not work:
  • Pressing Windows or Ctrl+Esc did not bring Start button
  • Restoring system to few previous dates
  • regedit can not be run. I got "registry editing has been disabled by your administrator"
I first needed to figure out how to enable regedit. I did some browsing on my Fedora laptop. I used Fedora to fix problems on XP. Nice!!

Fortunately, Ctrl+Alt+Delete works. I could run explorer and some other commands. I did:

  1. New Task in Task Manager
  2. Type GPEDIT.MSC and Press Enter
  3. Go to the following location
    • User Configuration
    • Administrative Templates
    • System
  4. In the Settings Window, find the option for "Prevent Access to Registry Editing Tools" and double-click on it to change.
  5. Select Disabled or Not Configured and choose OK
  6. Close the Group Policy Editor and restart your computer
Ref: http://www.pchell.com/support/registryeditordisabled.shtml

Now that I was able to run regedit, I needed to figure out which registry enabled start button etc. There were many choices which seemed like would do the job. I am a very impatient guys so did not feel like manually enabling those. I resorted back to more searching and found http://www.kellys-korner-xp.com/regs_edits/nodesktop.reg. Running this script and restarting it solved the problem. Funny thing is that I could run this script simply by hitting the URL and Selecting Run in IE. Interesting, isn't it?

By the way, http://www.kellys-korner-xp.com/xp_tweaks.htm has solutions for various XP problems. My appreciation goes to this site.

Thursday, July 24, 2008

Presidential Election and Oath Taking Drama in Nepal

If anybody correctly predicted the presidential election outcome of Nepal, please let me know the name and I will take a big bow. Not only the outcome is completely unforeseen and to the extent irrational but also the political drama that went along with it.

Nepali diaspora is abuzz with how both president and vice president are Madheshis. It's also abuzz with vice president wearing Dhoti/Kurta and swearing in Hindi. Let me first talk about both of them being Madheshis.

A day before the last day of nomination, I, like most of Nepalis did not even know about Parmanand Jha. All of sudden he was a VP candidate with a fair shot. There came three Rams, all Madheshi, as presidential candidates when it was supposed to be Nepal, Koirala and Prachand. To me, Rams and Jhas are nothing but the result of egotistical clash of these political chieftains. Prachanda does not want to see Koirala as pres, Koirala does not want to see Prachand as pres, Nepal can not agree with Maoist on all terms. So they are like, "Look, we are not going anywhere, let's make best out of it. We lost big time in Terai during election so why not nominate Madheshi as our party candidate. This will let parties from Terai (MJF and TMDP), vote in our candidate's favor. This will not only show that we care about their cause but will also split votes among different parties in next election as a result MJF and TMDP will not be as big as they are now." If I am mistaken on my analysis, give me any other valid reason why the big three parties each would nominate Madheshi as their presidential candidate? After all these are the same people and parties which failed to recognize the very agreement on Madheshi's demand they had made in February this year. I am a Madheshi so why am I cribbing about it? Should not I be rejoicing the fact the first Pres and VP are both Madheshis? Unfortunately, I am not very happy about it for a President should be Pro-Nepali not Pro-Madheshi or Pro-Pahadi, he should be respected by all sect of people and the foremost, he should have a vision and wisdom to unite people. I would have liked to see a name which Nepali people relate to as being a Nepali rather than a Madheshi or Pahadi.

The next big event was VP Jha taking his oath in Hindi. I am a Jha so I would rather he take his oath in Maithili which is native tongue of most of Jhas but I would have liked him to take his oath in Nepali, the official national language. Any other language, be it Hindi or Maithili or English, it's all the same for speaking in Hindi does not make anybody pro Indian as speaking in English does not make anybody pro-American or pro-British. To make a fuss about Hindi is making more mockery of oath taking ceremony which could not have been more farcical. Is not there usually a protocol that need to be followed for these events? What does current Nepali constitution say about it? If constitution and protocol don't allow anybody to take oath in Hindi then Mr. Jha needs to take oath again otherwise people need to stop making big deal of this. I agree it's more of a symbolic thing but one should judge him and the Pres not by the language they use to take oath but by their deeds to make Nepal a healthy and prosperous republic country, by their contribution towards creating a constitution which we all Nepali deserve.

I think it's a mockery of CA election that the party which bagged the highest number of votes does not have it's candidate as Pres or VP. I hope it does not get worse. I hope that next government is headed by Maoists. I hope that Dr. Ram Baran Yadav and Parmanand Jha understand the role of Pres and VP respectively. I hope that CA members understand that their role is to help write a constitution, not to be part of power sharing game. I hope that Nepal gets a constitution which treat all Nepali equally and all Nepali have equal opportunities regardless of their ethnicity. I know that my hopes are wishes of all of you and may your wishes come true.

Thursday, July 10, 2008

How to configure dual boot with Fedora 9 and Windows XP?

There are already tons of articles, howtos on this topic but one more does not hurt. My situation was kind of different as well.

One of my colleague left the company and I inherited his Lenovo T60 x86_64 laptop. The machine had Fedora 8 installed in it and I needed to setup dual boot with Windows XP and Fedora 9. Of course the easiest would have been to upgrade to FC9, partition the disk with gparted and install Windows XP on new partition. But nothing in life is this easy so why this should be. I had a previous experience of installing dual boot on Dell desktop way back in 2000 with Windows ME and Red Hat version 6 IIRC. But back then I had bought additional hard drive, one for each OS. Anyway, following is what I ended up doing in this case.

For some reason gparted could not partition my hard drive (dev/hds). It kept failing and error was not very descriptive. Most of these articles suggested that it's much easier to install Fedora on machine with Windows than other way around. With Microsoft, you should expect this anyway. When I tried to boot from CD with Windows XP, it did not boot. I think XP was confused with linux partition and it did not know how to proceed. This meant that I had to reformat the whole hard drive. Since I did not care about data that were already there, I could freely reformat it. The easiest way to reformat in Fedora is to sudo as root and do :

dd if=/dev/zero of=/dev/hds bs=521 count=1

This command deletes partition table and XP installation CD wont see any Linux distro in harddisk. If you restart, your BIOS should not see any OS.

I restarted with XP installation CD on it, XP formated the whole disk with NTFS and installed itself. Now comes the fun part of installing Fedora 9. I prefer Fedora Live CD than full CD ISO or DVD ISO. I rebooted with FC9 live cd, connected to wireless network in case I needed to download something explicitly and selected the option "Install to hard drive". After hitting next button couple of times, it gives you an option to how to install to hard drive. I chose 3rd option which allows to partition hard drive. I shrinked existing partition to 50G so that meant 50G will go to XP and the rest to FC9. Follow next couple of times, Fedora does its things and installs itself.

When I restarted, the first screen says Fedora, hit enter and then GRUB bootloader screens shows up. It gave me two options. One said Fedora and other said "Other" which is Windows. I think you can rename "Other" to Windows so that it's clear but I did not care. Selecting "Other" takes you to the familiar XP startup process.

Overall my experience was not very painful.

Monday, June 9, 2008

I so wanted to be wrong but...

About two and half months ago, my colleagues at work walked to a near by restaurant for lunch. We passed by a gas station and IIRC the gas price for unleaded (the lowest grade) was about to approach 3 dollar and I out of nowhere said, " I predict that gas price is going be $4/gallon by the first week of June." As time passed by I was getting closer to my prediction and obviously I was not happy about it. Today the average gas price is $4/gallon or more. Everyday I hear and watch how people are having hard time buying dairy products and regular groceries.

There are many factors that have caused this and I am not going to talk about obvious ones such as demand/supply, dollar getting devalued etc.

I believe it was year 1999 or 2000 when sales of SUV surpassed sales of compact cars in USA. I think it was the same year when sales of salsa surpassed sales of ketchup. I don't remember many news analysts talking about the long term impact of this shift in America's vehicle choice. It was proud endorsement of "Everything is big in America". I have seen many times in little city parks that parents wait in car with ignition on while their kids play. I have seen folks circling around gym to find a parking spot nearby the gym gate when they could easily get a spot in nearby spaces. So, what's the point? When people have abundance of resources, they tend to use them without any inhibition and without thinking even for a bit where all these lead to. I think wisdom is not to live in present only but to think about future as well. One should try to have a living standard which they can at least maintain throughout their life. You don't want to ride Mercedes one day and a Kia other day.

These days I read that the demand for fuel efficient cars have gone up. People are no longer looking for space but for fuel efficiency. Only sad part is that they had to be pushed by rising fuel price to change their riding habit.

Rising fuel price has ripple effect on price of almost everything else that you need in life. I am primarily concerned about food price. I heard Miss Condy Rice and President Bush saying that price of food in America have gone up because people in India and China are able to afford more food and they are consuming more. I do not deny this observation but I wonder about mentality and citing that as cause of rising food price. I don't care whether American people eat a lot or not. You eat based on how hungry you are even if you have more than you can eat. But I have observed in many restaurants that myself and my American friends can not finish all food in most of lunch plates. We simply throw them away. Why not have less quantity to begin with? Some food for thought.....

My apology to all my friends and American people who do not need to be told these things. I have many friends who care about environment, who care about rest of the world but I wish majority of people were like them.

Monday, April 14, 2008

Price of overconfidence and selfishness!!

One usually gets more benefits as they get more experience in their field. This usually and in most cases rightly is based on the assumption that more experience allows one to see a bigger picture in current and future context. One matures as they learn from their mistakes and is less likely to make similar mistakes. This does not apply to politicians specially if they happen to be Nepali politicians.

The constituent assembly election in Nepal which got postponed twice and finally was held on April 10 had such a denouement that outcome not only stunned the people who lost but also the people who won. Majority of so called experts had predicted the stalwarts from Congress and UML would hold their forte with Maoists grabbing some seats in remote areas. These experts just got big slaps in their face because Maoists are certainly heading towards a simple majority at the expense of the stalwarts. Is this a fluke? Here is a an effort in sensible rationalization of the outcome with top five reasons in ascending order.

1. Postponement of Elections: I believe that elections got postponed twice not because it was not feasible but because Maoists were not ready. They knew that they only had influence in remote areas. If elections were to be held, they would mostly win in those remote areas which could easily be blamed on fear and coercion. So they bid their time, gathered enough momentum in rest of the countries and once they were assured of some support in non remote areas of the country, they agreed for poll.

2. Fear: Just before the poll, Maoist chieftain made an statement that if poll results were not in their favor, they would go back to uprising. People were already tired of killing of thousand of innocent people, daily strikes and uncertainty. Fear of going through the grind worked made people vote for them. Add to top of this the fact that some candidates could not even go to their constituencies for prepoll activities. See my previous blog.

3. Overconfidence: The stalwarts believed what election experts had to say. They stayed at their cozy homes in Kathmandu when they should have been spending time in their constituencies understanding the ground realities and common people's aspiration. They assumed that people would not want Maoists who simply were portrayed as bunch of goons.

4. Selfishness: This primarily applies to parties from Madhesh. Mahanth Thakur formed TMDP after being a member of Congress throughout his life. He came to realize that Congress was not doing enough for Madheshi people. This came too late (few months before the election) which shows lack of foresight on his part but there is a saying, "It's never too late." MJF came to limelight because they fought for Madheshi's right. I think what they have done for Madheshi people in last one year is more than what Congress/UML/Sadbhawna have collectively done in past 15 years. MJF and TMDP did have wisdom to think about forging an alliance for the election but effort failed because they could not agree on how many seats each should get. To me it's very simple if you put Terai and Madhesh' interest before your party. But they all wanted a bigger share of the pies which ultimately resulted into none of them getting enough share of pie to be called a force from Terai. I find more fault with Mr. Thakur.

5. Need for Change: People gave a clear majority to Congress in the first election that was held when King Birendra yielded to democracy. Congress squandered this majority and instead of focusing on people, the leaders focussed on themselves and split their party. Same thing happened to UML. People witnessed many permutations of Koiralas, Nepals, Deubas, Thapas and Kings during these years. None of them showed will to stand for people and hence the need for change. Maoists talked about bringing a change in very simple terms which common people could understand and results are there for all to see.

I always see the proverbial glass as half full instead of half empty. I believe that everybody should get a chance. Where it's a fluke or hard labor, Maoist have been given a mandate to bring a CHANGE in Nepal. They should grab this opportuny with both hands and show the whole world that they can walk the talk. I will certainly be hoping for that.

Sunday, April 13, 2008

what is a fair election?

It's about 2 AM in the morning and the more I try to sleep, the more thoughts come into my mind. Instead of flipping in my bed umpteen times, I decided to let it all out over here.

Nepalese people participated in a historical election on Apr 10. Yes, it was historical because Maoist guerrillas who have been waging a so called "people's movement" decided to lay down arms and join democratic political process. Whether it was really a people's movement that's a topic for different blog but let's talk about this election here.

According to Nepalese Election Commission, there was about 60% polling which by any standard is a success. But the commission as well as some foreign observers affirm that poll was fair except some booths where some irregularities were observed. As of today, the commission has decided to do a re-polling on 70 of booths. But what is a fair poll process? I believe in KISS principle so to me a fair poll process is process where each concerned party has equal opportunity to share their manifesto and future plan with general people AND people vote without any fear or coercion.

It's very well known fact that some of candidates (Surya Bahadur Thapa in Dhanusha for an example) could not even hold a rally in their constituency because Maoist or other parties did not let them. Poll was supposed to start at 7 AM in morning in constituency number 6 of Sarlahi district and booth was closed without a single person voting because all ballots had already been stamped. I am sure these people formed part of 60% that the election commission projected. In terms of security, there were one police with a gun and three polices with baton in most of booths as per my conversation with friends whom I talked over the phone on the day of voting. Commission proclaimed that they were fully prepared for the election. What the heck?

I sympathize with the commission that they did not have enough time to prepare. So why was the election not split in two or three days if there were not enough resources??

What bothers me the most is so called independent observers including honorable president Mr. Carter. Carter center called the election remarkable and excellent. My question is what benchmark and data set were used to come to this conclusion? As per my conversation with folks across different constituencies in Terai region, they did not come across any foreign media and observers. My question to the Carter center and to Mr. Carter is this: Is visiting polling centers in Kathmandu and Pokhara enough? Did you consider visiting areas which were known to be problematic to assess preparation? Is this what expected from the Carter Center? Can somebody fill me with the details please?

Off to bed....