Maven archetype for testng + selenium

Lately I have been working a lot on front end stuffs such as JBoss Portal and Richfaces although my preference has always been to work on back end. Well life is full of these conflicting choices, isn't it?

I write sample applications for these JBoss Web Interface projects and testing them are included as part of continuous integration testsuite. To help with automation and to give easier ramp up time for new team members, I created a maven archetype for writing test cases using TestNG and Selenium. I purposely decided not to add cargo stuff as most of time we have preconfigured application and web servers. To use this for your own project, here is what you need to do


mvn archetype:generate -DarchetypeGroupId=org.jboss.maven.archetypes -DarchetypeArtifactId=selenium-testng -DarchetypeVersion=1.0 -DgroupId=org.whatever.project -DartifactId=myproject -DarchetypeRepository=http://repository.jboss.org/maven2/


This has .classpath and .project for Eclipse/JBDS projects as well so you can import the generated project into Eclipse/JBDS and your project build path will automatically be set correctly. This was my first real foray into maven and I still think that maven needs to mature more in terms of usability. I hear good things about Maven 3 so I am looking forward to that.

Bug fixing: Two Steps Forward and One Step Backward

Currently I am leading the testing effort of couple of software frameworks which have fairly good user base. They are frameworks because they allow software developers to build a software product as opposed to provide an out of box software product for end users. While fixing bugs and testing them, all of sudden, I realized that bug fixing overall does not necessarily improve the inherent quality of software whether it's a software framework or software product. This self realization some how had a theoretical familiarity to me.

Going down the memory lane, I had read few good books on software design and development as well as software project management as part of my preparation for job interviews when I was about to graduate. One such book was The Mythical Man-Month by Mr. Frederick Brooks, JR. Back then it had made some sense but it it was one of many books that I had read. Its significance and relevance might have fizzled in elegance of other good books that I read notably GOF book on design patterns. I revisited Mr. Brooks literary creation and behold, it talked about exactly what I felt in chapter 11. ( I did not think Mr. Brooks had any pun intended when he talked about bug fixing and maintenance cost being 40% of development cost. These days I hear too many chapter 11 instances in financial news ;-) ).

In this chapter, Mr. Brooks describes bug fixes as two steps forward and one step backward. Here are the main points on why:

• Fixing a defect has 20-25% of change of introducing another.
• Whole set of regression testing is very costly.
• All bug fixes tend to destroy the structure, to increase the entropy and disorder of the system. Less and less effort is spent on fixing flaws introduced by earlier fixes.
• Things are always at their best in the beginning. Bug fixes increases disorder of the system and users get more sophisticated
• Software Development is an entropy decreasing process, hence inherently metastable. But maintenance is entropy increasing process, and even its most skillful execution only delays the subsidence of the system into unfixable obsolescence.
  

Given that Mr. Brooks and many others observed these so many years back and I observe it as well, I wonder how much improvement we have made in terms of software project development as a whole. May be it's because of the fact that the only constancy is change itself and designing a system for change is more widely discussed in the literature than practiced. Mr. Brooks, you definitely make more sense to me now than what you did four years back. Not your fault though.

JSR-286 talk at Austin JUG

I will be giving a talk/presentation at Austin JUG about new features in JSR-286. The details of talk at http://www.austinjug.org/index.jsp?p=events-20090224 .

Technical Review of a book on JBoss Portal

I had the pleasure of being one of the two technical reviewers of a book on JBoss Portal. The details of book is at http://www.packtpub.com/jboss-portal-server-development/book. This book is another example of world being flat.

I was electronically contacted by publisher who till date I have not met and don't even know where exactly it is located. I electronically signed the contract. I got book chapters one by one through a mediator who was liaison between me and the author. I have not met the author yet and book is published.

I hear that if publishers get break even with technical books then it's usually considered a good thing. This also means that a technical reviewer does not usually have a monetary benefit and my case is no exception. I did get two free copies of the book and most importantly a very brief bio which is printed in "golden" letters. And, I am grateful to publisher and author for that. Now back to book.

I believe that book is a good introduction on Portal servers in general and when to use them. You will also get a good feel of different features such as CMS, Security, Management of JBoss Portal server. It has working examples of how to use JSP, JSF, AJAX in a portal environment. The book ends with a good introduction of JSR-286 aka Portlet 2.0 specification.

So, if you are curious about Portal world and looking for a spec compliant open source portal server, then this book is a good to have item in your bookshelf. If you find something not working as explained in book then feel free to comment over here. I will hit myself for a while but make sure its errata is updated accordingly. If you find everything working as it should, I wont mind seeing few words of appreciation. ;-)

We got electricity. Finally !!

Most of the world is agog with upcoming Obama's inauguration and I also belong to that part of the world. But I have something else to cheer for which depending on how you look at is equally exciting.




Sisautiya, the village in Nepal where I was born, finally has electricity. Yes. Yes and Yes. This is the village where I spent the first 16 years of life and went to a nearby high school which also did not have electricity. I hope my grand parents have saved the lantern that I used.

Friedman's 10 World Flatteners

Thomas L. Friedman's "The World Is Flat" is probably the best non fiction book I have read so far. It gives an amazing insight into 21st century: starting from why things are the way they are, how they will be and what factors will shape them. Here the "flat" is in reference to level playing field where each nation has almost equal opportunity to be a player at a global level. He provides a list of ten forces that flattened the world.

1. Fall of Berlin Wall (Nov 9, 1989)

• It was not only blocking our way but was blocking our sight. World was sort of like a frog in a well. Demolition set the frog out of the well.
  
• End of communism which in the simplest term is "a great system for making people equally poor".
• Created a chain reaction of freedom.
• Simultaneous birth and popularity of IBM PC/Microsoft Windows
  

2. Birth of Internet Browser Netscape

• Created a dot com bubble which helped corporation realize the power of internet which ultimately resulted into huge investment in connecting the world through fiber optic network.
• Dot com bust lead into cheap availability of these fiber optic data transmission network which third world countries could use.

3. Work Flow Software

• Automated communication between different departments of a corporation. New standards for software to talk to each other (SOAP, XML for Web services etc)
  

4. Open Sourcing

• Tell me about that. ;-)

5. Outsourcing

• Definition: To take a limited function of a company that was being done in-house such as research, call centers and have another company perform the exact same function and then reintegrate their work back to overall operation.
  
• Dot com allowed country like India, China and some Eastern European countries to use all the data line practically for free.
  
• World could use India's millions of engineering brain power not only in software but in accounting, medical data analysis, customer support
  

6. Offshoring

• Definition: When company takes one of its factories say in Canton, Ohio and moves the whole factory offshore to Canton, China.
• China wanted to attract foreign manufactures not simply to manufacture 1 billion pair of undies for sale in China but to use low-wage Chinese pairs of undies to sell 6 billion pairs of undies to the works at price that were a fraction of what companies in Europe and America.
• Companies started breaking their business into three parts: part to do in China, part to sell to China and part to buy from China.
• China flattener has been wrenching for manufacturing workers but a godsend for all consumers. By mid-1990s alone, cheap imports from China have saves U.S consumers $600 billion and saved US manufactures untold billions in cheaper parts for their products. This saving in turn helped Federal Reserve to hold down interest rates longer, giving more Americans to finance home and new business.

7. Supply Chain Management (SCM)

• Wal-Mart's uber cool efficiency on SCM. Its system is inter connected to such an extent that the moment a customer purchases something, the record goes to all the way to manufactures through corporate headquarter and wholesale dealer without a single intervention from a person.
  

8. In Sourcing

• UPS story: When you return a Nikon camera, it does not go to Nikon headquarter all the way. UPS has certified engineers to fix the camera. If they can not fix it, they ship a new camera to customer thus saving on extra shipping that would have been needed otherwise.
  
• Def: a new form of collaboration and creating value horizontally.
  
• Even small companies can do business globally without opening a shop in rest of the world. UPS acts as your global supply chain manager.

9. In-Forming (easy access to knowledge - Think Google)

10. Influx of Gadgets - Digital, Mobile, Virtual

• Being able to print boarding pass at home
• Blogging from anywhere through various devices - even mobile phones.
• Engines talking to computers, talking to people, talking back to the engines, followed by people talking to people - all done from anywhere to anywhere.
  

BTW, above is only one third of the book.

Lessons Learned from Scalability Testing

Currently I work on JBoss Portal project and recently I have been working on its performance and scalability part. The outcome of my recent effort is published at this blog entry. This was a very prolonged testing which required lots of patience, working with several teams, learning many new different tools and concepts. But now it feels like every bit of it was worth it. Here are the lessons that I learned not in any particular order:

• Performance and scalability testing is not easy. Besides it being heavily technical I believe it's an art and you mature by experience.
• Set a goal first otherwise there is no end to this. Be very concrete: I want my application to handle x concurrent users with response time of less than y second/s and it should scale more than z%.
  
• If the application/system you are trying to improve the performance of and making more scalable is not developed by you, it's a must that people who developed are aware of scope of your work and helpful. I had good fortune of this being the case. I do not even want to imagine how it would have been otherwise.
• The best way to find bottlenecks is taking several thread dumps and taking very close look at them. I found it the most revealing and helpful than any profiler (commercial or open source). TDA is a good tool to analyze thread dump. I used three profilers: JBoss Profiler, JProfiler, SAP's Memory Analyzer.
• Regardless of what people say, open source load testing tools serve most purposes. I found Grinder to be very straightforward, not that hard to configure, and very vibrant community.
• For managing distributed deployment, use or extend a distributed framework. I used Smartfrog components to manage instances of JBoss AS, start up load generator, start up apache web server and load balancer otherwise I would have had to log into around 15 servers to start/stop/configure these components.
  
• Try to optimize most common code execution path. Optimizing code base which crawls but is also used rarely wont help you much.
• JVM's default garbage collection configurations are pretty good. I have toyed with several configurations and none of those seem to have any significant impact in my use case. My point is that you play with GC configurations when you have fixed other problems.
  
• JConsole that comes free with Sun JDK is pretty good tool to observe GC behavior, memory footprint and different logical memory spaces (eden, perm etc) of heap. Use that as a first tool for your profiling.
  
• It's a common knowledge that database is always a bottleneck. Hibernate's show_sql switch is a very good friend. Make sure that you do not see queries being executed more than they should. Tuning those bumped up performance quite a bit.
• The first thing you turn off after you have done basic optimization is logging. It has a big overhead simply because logging requires write to file and disk access.
• Don't use NFS for testing.
• If servers are part of cluster, make sure that they all are part of same subnet and secluded from any other network traffic.
• Find a compromise between loading everything in memory and loading everything on demand (lazy loading). The former may show a good performance in the beginning but crawl once memory fills up and lot of full GC starts happening. The latter will have higher DB utilization. These settings are dependent on nature of your application and nature of how the application will be used. There is no universal setting. Let me know if you find one.
• This blog is a WIP. I will add more bullet points as I think of them. :-)