Le mercredi 2 décembre à 11h00 - heure de Paris - Sun Starup Essentials vous invite à découvrir l'offre de Planet-Work, notre partenaire hébergement.

Frédéric Vannière, Directeur Technique, vous expliquera comment il a mis en oeuvre les fonctionnalités de virtualisation de Solaris et de ZFS pour proposer des packages d'hébergement souples, évolutifs et économiques. En route pour le Cloud Computing !

Inscrivez-vous !

I had an interesting discussion with 2 colleagues about the possible interest of putting the swap space of a system on a SSD.

If I consider the gain in latency that an SSD brings versus a capacity disk - in the region of 100x - the solution seems obvious. Swapping - or more precisely paging - must be much faster/ with an SSD. Since RAM is expensive versus SSD, I could even be tempted to design a system with a small amount of RAM and a large amount of swap space on SSDs. In other words, I can ask myself if trying to prevent my system to page is still a good fight?

Let's try to shed some light on these questions.

Paging takes place when my system runs out of RAM because more processes are created or because existing processes requires more memory (check this article for details about how to monitor paging). At some point, the operating system keeps on looking for pages in RAM that can be transfered into the swap space, while at the same time it brings back in RAM pages that were paged out and that are required by running applications. This situation is commonly referred to as a paging system. At this point, the performance of my system brutally goes down: copying memory pages back and forth between RAM and disk slows down my whole system mainly because of disk performance. Moving the swap space from a disk to an SSD does not reduce this activity. It only makes it faster. Bare in mind that the CPU doesn't have direct access to the swap space, to the SSD. For the CPU to access data or instructions that have been paged out, these data or instructions still need to be copied back into RAM, which brings us to another side effect of paging: it creates traffic on the IO bus.

In addition, ahead of the critical paging situation, when the demand for RAM starts to grow, other things happen on my system. I am using ZFS for my storage and ZFS has its primary cache - the ARC - in RAM. When RAM gets under pressure, this cache sees its size reduced. The data removed from the ARC goes in the ZFS level 2 cache - the L2ARC. The L2ARC can be located either on disks or SSDs, but as soon as it is involved there is some additional traffic on the IO bus that now competes with the traffic created by the paging activity. Eventually, when the L2ARC gets full, the data is not cached anymore. A long story short, if I am running an application that creates a lot of IOs the shortage of RAM impacts its performance.

Finally, we compared the performance of an SSD versus a disk, but in terms of latency an SSD is still 1000x slower than RAM, so the impact that paging brings (i.e. moving from RAM to SSD) is still noticeable. At the end, even though SSDs can improve paging performance, preventing my system to page is still a must if I want to get the best out of it. If I have some extra money to spend on performance after increasing the RAM of my system and if my application is IO intensive I would rather buy an SSD for the L2ARC cache rather than for swapping. This will certainly have a positive impact on the IO perf of my application.

Today is Nov 21th, 2009. News shorts of interest to our communities, including: New date for EU review of Oracle/Sun acquisition, getting closer to v3 fcs, new OpenESB and OpenDS releases, Devoxx whiteboards, new customers and japanese event, and more.

Waiting for Godot
I read Waiting for Godot for HS, but I didn't expect to live it...

• E.U. Takes More Time to Review Oracle-Sun Deal

On the Road to GlassFish v3
We are getting very close. The buzz around JavaEE 6 and GFv3 at #devoxx was very positive; some more links:

• Migrating your Apps to JavaEE 6 (Wouter van Reeven)
• Adam Bien on
  • Adoption of EJB and JavaEE, Ready for Production and ... (it is) a massive rewrite.
  • Also see Adam's JavaEE Patterns and Best Practices @ Kenai, all based on GF.

New Releases
Final and Release Candidates releases:

• OpenDS 2.2.0 Release Candidate 3 is now available
• DSEE 7 Download, Documentation and Upgrade Guide Available Today
• JavaCAPS 6.2 is now available (ontherecord, webpage) - Based on GlassFish 2.1p2 and OpenESB 2.1
• Fuji (OpenESB v3) Milestone 9 is now available - (Mark's note, Wiki page).

More Devoxx
Devoxx is over.  By all accounts, a successful show.

• Devoxx 2009 picassa album... with
  • Whiteboard pics - JVM, CI Tool, Persistence Layer, Build Tool, UI Framework
  • Hudson #1 in CI (113); next are TeamCity (31) and CruiseControl (30).
• Notes on Mark's jigsaw prezo (from Reinier Zwitserloot)

GlassFish Customers and Events
New customers; new events

• Mapion Co. Ltd, in Japan (chhandomay, customers) - Using GlassFish Enterprise Server, MySQL Cluster and more.
• Health Sciences South Carolina (ontherecord) - Using JavaCAPS 6 including GlassFish ESB, MDM, HL7, LifeRay and more.
• New event: Benefits of GlassFish and MySQL in Japanese (Dec 9th, 2009).  All events at GF Event Calendar.

Other News

• ICEfaces 2.0 is now available for download.
• State of OpenSource AppServers by Masoud Kalali
• The LHC is back online! (website, #LHC)

A new Eclipse proposal (Eclipse Development Process: Pre-Proposal and Proposal) has just been posted at Eclipse.org.

Quoting from the proposal, the scope of the Gemini project is two-fold: • Integration of existing Java enterprise technologies into module-based platforms; and • Implementation of enterprise specifications for module-based platforms The initial emphasis is on standards developed by the OSGi Enterprise Expert Group. Gemini is organized under 6 subprojects, each seeded with contributions from SpringSource or Oracle and the overall lead for the project is Mike Keith. The project mentors are Wayne Beaton, Doug Clarke and Adrian Colyer.

A summary of today's news of interest to our communities. Today is Nov 19th, 2009. One more day to go at Devoxx, some Terracotta news and more GlassFish Events. The Java EE 6 specs are in voting right now, and we are still awaiting Godot. Note - this is an experiment to flush out the daily news that otherwise we can't cover due to limited time. Let us know how the format works for you.

Terracotta News

Bumped into Alex Miller's blog and it has several posts worth mentioning:

• Terracotta Acquires Quartz
• Terracotta, Ehcache, Hibernate, and more to come
• Terracotta Use Cases

Devoxx Updates

• Closures coming to JDK 7 - Samples from Cay;reactions from AlexM and StephenC.
• JDK 7 schedule extended til Sept 2010.
• Jigsaw downloads available
  • And, from last week see JDK 7 M5 and companion NB 6.8
• Hudson, the most used CI Solution at Devoxx (@rob2)
• Photos (from Picassa Album)- Happy faces: Roberto; Antonio and AlexisMP - one, two; Ludo.
• Danny's summaries on Day 1 and Day 2. Many, many tweets at #devoxx.

New GlassFish Events

• Added JSF Summit, Sao Paolo Tech Days and JSF Days 2010 to the GF Event Calendar.

Calling all GlassFish-related events!

We maintain a master calendar for events related to all the projects in GlassFish Portfolio at Google Calendar; if you are hosting such an event, or presenting at one, please let us know to theaquarium at sun dot com.

The calendar ID is 3722ulvfgor2qabrut1mkia5m0@group.calendar.google.com, and you can access it in a number of modes: • RSS Feed • iCal Format • HTML

We are pleased to share with you another initiative that could be of interest to Startups such as yours. Our engineering team alongwith our partners will conduct a fortnightly technology session on a topic of interest to startups. The inaugural  session is scheduled for 25th November at Divyasree Chambers in our Sun offices in Bangalore and the details are as below:

Technology Session on My SQL - High Availability Solutions

Where : Sun Microsystems Office , Divyasree Chambers , Bangalore
When : 3pm to 6pm on Wednesday , 25th November, 2009

Agenda

Welcome

3.15 PM - 4.15 PM  : Different High Availability Solutions in My SQL

4.30 PM - 5.30 PM : Scaling with My SQL

               + Challenges faced by end users to keep servers
                 performing at their best while being configured to be
                 highly available.
               + Sun technologies addressing the challenges
               + Brief overview of both MySQL & Sun availability options
               + How to  achieve end to end high availability without
                 compromising on performance and scalability.

5.30 -5.45 PM : Closing Remarks

There are only 30 seats available , if interested please send us email on isvsupport-AT-sun-DOT-com  with the following details:
Email Subject: Technology Session on My SQL - High Availability Solutions
Name of the participant :
Functional Area
E mail
Phone

Look forward to hearing back from you. Also would appreciate inputs on the technology topics that would interest you, so that we can plan subsequent technology sessions.

Désolé, il ne reste plus de place pour cette matinée dédiée à l'optimisation des performances MySQL.

Date : 24 novembre 2009

Session 1 : 9h30-10h45

• Architecture MySQL : Parsing, exécution, optimizer, query cache, binlog…etc
• Architecture Innodb : Clustered index, bufferpool, hash index, insert buffer, locking model, MVCC, recovery log, checkpoint…etc. Comparaison avec les autres moteurs de stockage (PBXT, MySQL Cluster) et bases de données (PostgreSQL, Oracle ..).

Pause/Café : 10h45-11h00.

Session 2 : 11h00-12h30

• Évolutions récentes : Google SMP patches,  XtraDB, Oracle innodb plugin 1.0.4
• Apports de MySQL 5.4
• Améliorations à venir :  Ahead flushing / Adaptive checkpoint,  Purge lag / Purge thread, Splitted locks, etc…Explication technique des améliorations et des gains de performance obtenus.

Nous parlerons aussi de MySQL Enterprise et du MySQL Query analyzer qui vous permet d’optimiser un environnement MySQL.

Cette matinée se veut conviviale et interactive avec Q&A.

Check out this photo of Steve Harris's keynote at Devoxx; Sparky has a new friend! Twitter is full of positive comments. I need to do a pass to separate the interesting ones, but raw data at: #glassFish OR glassfish, #JavaEE6, #glassfish and #Devoxx. Looking forward to a complete report from Alexis (this photo is from the standing-room only JavaEE 6 University talk that he and Antonio gave). PS. Thanks to whoever took Steve's photo. I'll add attribution as soon as I find the author Aaron Houston for Steve's photo.

The adoption of Hudson continues to grow and we are also beginning to see movement in Sun's commercial offering.

Kohsuke's Hudson Feed has many examples of adoption. The community is very active and recent posts include A Campfire Plugin and a CMake Plugin, a spanish Tutorial on SVN and Hudson and Using Hudson with Rational Team Concert (from the official IBM site!). Two posts with nice, quotable compliments are:

• From Grant Smith's Creating a Continuous Integration Server for Java Projects Using Hudson at Wazi: our conclusion - based on a combination of first-hand experience and other people's opinions - is that Hudson is the current front-runner among open source CI engines. • From Joe Heck: Hudson - A lot of things done right: check out Hudson. It's a quick check - one download, one command-line, and you are rolling.

Three examples on the commercial side, all from today:

• We closed another customer for Sun's Hudson Support; nice! Just let us know if you have any questions.

• We also got email from one of Sun's Principal Field Engineers. He is overseas visiting a (very) large customer where he found wide use of Hudson and strong interest in improved support. We not even knew this company was using Hudson - strike another win for Open Source!

• And, our friends at Sun's Inner Circle - Sun's newsletter for CIOs - included an article on Hudson in the Nov/Dec issue.

This past week I have been working with 2 Web2.0 startups - members of Sun Startup Essentials, they get the support for free - that are looking at improving the response time of their web site.

We've been working at understanding how they servers behave today and what could prevent their application to scale, which led me to show how to run a quick diagnostic on an existing system and to restate some key fundamentals in terms of scalability.

Let's start by the fundamentals.

Any piece of software is nothing else than a set of instructions that need 2 things: computing units to execute on, and a fast access to data.

Computing units are available in the form of a hardware threads located on a processor. The number of threads that a server makes available to the application depends on the number of processors on the server, the number of core per processor, and of threads per core. The only thing we have to keep in mind is that the industry has moved heavily to processors that provide many hardware threads (up to 64 on a CMT chip), so taking advantage of multiple threads is definitely the right way to scale as far as computing power is concerned.

As per the fast access to data, the rule of thumb I always used is that good performance can only be achieved if the data is located in physical memory - in RAM. As soon as disks are involved the penalty is in an order of magnitude - if not more.

How do I take advantage of multiple threads with MySQL or memcached? The good news is that both support multithreading which means they both have the ability to create many software threads that are mapped to HW-threads to execute many instruction sets in parallel (with memcached you may have to recompile the code according to this page).

Now how do I check what's really happening on my system?

First, let's check how many HW threads are available on my server. On Solaris or OpenSolaris I run psrinfo:

root@pelvoux:~# psrinfo
0    on-line   since 11/15/2009 17:09:45
1    on-line   since 11/15/2009 17:09:49

The command reports me that I have 2 HW threads on line (I have a dual-core X86 processor on my laptop).

Then I can check how many software threads each application is actually using. To do so I run the prstat command. On the following example prstat 5 tells me that MySQL is using 10 software threads. The number is displayed on the right-hand side column NLWP. Note that with 10 software threads MySQL can not take advantage of a server that has 64 HW-threads, but for my 2 core laptop this is already more than needed.

Next, are the software threads leveraging the HW-threads? Are they keeping the CPU busy? The fastest way to check if the CPU is loaded is to run vmstat 5:

In this example my CPU spends 98 to 99% of its time idle, and 0 or 1% executing code from an application - column us, or from the system - column sy. If MySQL was running on all CPU cylinders, I would see most of the load reported in column us, and possibly a few percent points in column sy. In my case MySQL does not receive any request.

But there might be other cases where I see both slow or hanging requests and a CPU partially loaded with applications. If my CPU is not fully loaded, and if I have enough software threads, why do I see slow requests? Well, in that case it's likely that requests are competing for resources - such as tables - that can not be shared. This is a situation that impedes scalability. Only a few requests are executed at a time keeping only a few threads busy. Though outside of the scope of this article note that if you are using MyISAM, moving to InnoDB can help you to address table-lock issues, hence improve the scalability of your application.

So with psrinfo, prstat, and vmstat I can have a quick overview of how my server is doing in terms of CPU power an load.

Now, how can I check if I am running out of physical memory?

The diagnostic above assumes that the limiting factor is the computing power and not the access to data, not the amount of physical memory. But how can I be sure I have enough RAM and what happens in the face of a shortage?

When running out of RAM, a system starts to page. It tries to free some space in RAM by transferring some content from RAM - available as pages - to disk. The OS kernel scans the RAM to locate the last recently used pages and put them on disk, with the hope that they won't be used anytime soon (in which case they would have to be brought back into RAM).

A peak in paging activity is nothing wrong. Cleaning up the RAM a little bit does not mean I am facing a shortage. It's when my system keeps on paging that I know I do not have enough RAM. Once again vmstat 5 helps me monitor the rate at which the kernel scans the RAM looking for pages to be moved away: the sr column reports the scaning rate. If it never goes back to zero it's time to add some more RAM to my server.

The last batch of JSR's for JavaEE 6 were submitted earlier this week for Final Approval Ballot . The ballot will start on 11/17/09 and end on 11/30/09. They are:

• JSR 316 - JavaTM Platform, Enterprise Edition 6 (Java EE 6) Specification • JSR 315 - JavaTM Servlet 3.0 Specification • JSR 317 - JavaTM Persistence 2.0 • JSR 318 - Enterprise JavaBeansTM 3.1 • JSR 322 - JavaTM EE Connector Architecture 1.6 • JSR-299 - Web Beans (now called CDI)

Several JSRs had been submitted and approved previously:

• JSR 314 - JavaServer Faces 2.0 (News@TA, vote results: 12 YES/4 Not voted)
• JSR 330 - Dependency Injection for Java  (News@TA, vote results: 14 Yes/1 No/1 Not voted)
• JSR 303 - Bean Validation (Emmanuel's note, vote results: 12 Yes/4 Not voted)

A few of the specs went through the lighter-weight Maintenance Process, including:

• JSR 311 - JAX-RS: The Java^TM API for RESTful Web Services JAX-RS 1.1 (Paul's note, change log)

GlassFish v3 is not just modular but the components can be updated through IPS-based UpdateCenter machinery. The Update Center team has been evolving tracking the IPS changes and adding refinements of their own; newer releases, like GFv3Preview, have been running recent UC, but the repositories for GFv3Prelude were running an older version of UC. Last week the Glassfish team pushed Update Center Toolkit 2.2u2 to the Prelude repositories. In normal conditions you should not notice the change but if you visit the repository directly you will see new graphics and additional facilities (like package search) plus improved performance and metrics.

GFv3 FCS is around the corner and we are seeing very high interest in the release. At that point we expect the bulk of the GlassFish downloads to switch to being served from the IPS repositories - and we will find out if our capacity planning has been accurate :-)

Migration time! OpenSolaris, NetBeans and Hudson have moved (part of) their infrastructure.

The OpenSolaris Website Community migrated opensolaris.org from an ad-hoc web app to XWiki on October 26th, 2009 completing phase 2 of the OpenSolaris.org transition. Check the Transition FAQ for more details. This move had been in the planning for a long time and is still unfolding. The NetBeans site moved the week of Nov 2dn to a new site, see the Announcement and the FAQ. The new NB site uses the Kenai infrastructure but is its own instance, separate from that of Kenai.org. I believe this move has completed.

The last (ongoing) move is for Hudson. Most of Hudson was at Java.Net but some parts were not - like the confluence-based wiki. After the availability problems from a couple of months ago, Kohsuke and the community decided to move the bulk to Kenai. That move is still ongoing but some key sections, like the front-page, have already moved.

In all cases, these moves are intended to be (mostly) transparent to the users (hopefully with improved QoS).

This week carries a new episode in the Sun/Oracle/EU saga: The EU Comission has issued a statement of objections on the acquisition of Sun by Oracle.

Since I am a Sun employee, I will just provide the basic links, no matter how tempting it might be to go beyond that... • (Nov 9th) EU issues SoO - I've only found indirect references to the SoO, like Sun's K-8 Filing. • (later that same day...) Oracle reacted with a PR Statement. • (a bit later...) The DOJ issued their own comment on the EU SoO. • (next morming, Nov 10th) And the EU comments on the DOJ comment (Reuters) • (and, on Nov 11th) According to eWeek Europe, the next step is a hearing on Nov 25th.

The material would make for good pulp fiction. It's very sad to see the impact on people, but today I was talking with a friend that was affected and he was being very good at keeping things in perspective, so I thought of using the front cover of a true pulp fiction: Doc Savage - which I first encountered in an old Spanish translation in a storage room in my grandfather's flat in Barcelona (together with copies of The Shadow and El Coyote).

Perhaps also time to watch again the movie? Blu-ray, pretty please?

Vaadin and ZK are both popular RIA frameworks that have been tested to work with GlassFish v3. The JavaDude has a detailed blog on "ZK 3.6.3 with Netbeans 6.8 Beta on Glassfish V3" (this is the ZK release from a few days ago). It discusses developing with or without the ZK community plugin for NetBeans (which is mostly about adding meta-data to a Java EE project), creating ZUML pages with a component palette and deploying to GlassFish v3. This framework does Ajax and push with no JavaScript exposed to the developer and should be familiar to people used to Swing development. Bobby wrote a blog entry you may want to revisit.

Vaadin is another framework that keeps the developer away from JavaScript. It builds on GWT and has some interesting OSGi features that make it a good fit for the GlassFish v3 modular architecture. Vaadin's Petter has several tutorials showing the use of the GlassFish servlet 3.0 implementation but also how the Vaadin OSGi packaging allows for various options to avoid having to carry the framework with the application. It also discusses having multiple versions of the framework deployed in GlassFish v3. If you're interested, start with this "Deployment Options on GlassFish v3" article.

Both ZK and Vaadin are GlassFish Partners, just like WebORB, ICEFaces (now in Alpha and tested to work in the most recent promoted builds of GlassFish v3) and many others.

This is the first of our weekly news catch-up and covers Nov 1 to Nov 11, 2009. This week the news catch-up is partial; next week I'll create the entry through the week and will try to be more comprehensive. This week we also cover old news on JRuby and OSGi.

GlassFish and Middleware News

• IBM DeveloperWorks: Introducing Metro. Nice to see IBM refering to the GlassFish WebServices implementation as it is Used by Many, including IBM. Incidentally, the article triggered several internal and external threads on the topic, see for example: Broken Fault Handling in Metro.
• Subversion is now an Apache Incubator podling and same for LibCloud (from Apache incubator)
• New IzPack, and new GlassFish v2.1.1 installer (via alexismp)
• Sun's Financial results for Q3FY2010 are now available. The revenue for MiddleWare+MySQL are shown as a single entry and are (slightly) up, Year-to-Year (Y/Y); Java and OpenStorage are significantly up Y/Y; other segments are down Y/Y. Also check the numbers for previous quarters.
• IntelliJ IDEA 9 (Maia) goes Beta with full JavaEE 6 (and GlassFish v3) suport.
• Sun's to cut 3000 jobs.
• New record month in GlassFish GeoMap numbers (more details still to come).

Predicting our Systems Future

• Systems-related: New DTrace CheatSheet, Simple LB in OpenSolaris
• Latest installment in Chris's series on Reduced Install of OpenSolaris. The series deserves a spotlight...
• Dedup in ZFS. This is a big deal; DeDup is big $$ business.

From the past: OSGi in GlassFish (triggered by this thread):

• Sahoo has done a lot of work on the OSGi portion of GF; see his blogs on: Running GlassFish on Equinox (and other OSGi Platforms), OSGi enabled web applications inGlassFish, Developing Hybrid (OSGi + Java EE) applications in GlassFish and Our second hybrid application: EJB as OSGi Service and OSGi Declarative Service bundle in GlassFish.
• Also see Arun's note building on this work at GlassFish v3 with different OSGi runtimes – Felix, Equinox, and Knoplerfish

From the past: JRuby on GlassFish (triggered by this thread)

• From GF Wiki: Getting Started with JRuby on GF

I've added a couple of new twists to the coverage of GlassFish News at TheAquarium to do a better job while controlling our time investment:

• I've started posting short news posts to my twitter feed as I encounter them;    also tweeting are Frank, Giuseppe, Andi, Binod and AlexisMP. • Nobody is tweeting at TheAquarium, but...    if you want to follow the tweets from the editors, use @theaquarium/editors. • We will continue doing the usual spotlights posts as time permits, • I'll do a weekly summary of news uncovered in the spotlights,

I've also started using ScribeFire to reduce the cost of creating posts, but this should be transparent to the readers.

Reporting on news is a losing battle, but I'm hoping that this approach will keep TheAquarium the best source for news on the (larger) GlassFish community while giving the editors a bit more "free" time to invest in other tasks.

The goal of the National Health Information Network (NHIN) is to provide secure, nationwide, interoperable health information infrastructure that will connect providers, consumers, and others involved in supporting health and healthcare. And the CONNECT Gateway is intended to let the federal agencies connect to the NHIN.

Within the HHS, the ONC is the main entity that coordinates these efforts and it just has choosen Health Information Exchange Open Source (HIEOS) as a key portion of NHIN Connect.

And, HIEOS - developed by Vangent - is using several of our OpenSource components - see Architectural Diagram - including OpenESB and GlassFish, and MySQL.

Here is the follow-up on the first ZFS compression article where I've been discussing the performance gain that compression brings and the way to measure power consumption with OpenSolaris.

With a simple benchmark consisting in copying/duplicating a 4.8GBytes file, I checked that ZFS compression improves the IO perf. I now want to measure the impact on power consumption.

During the 7m19sec that it takes to cp the large file without compression, my computer consumes 3.65Wh for an average power of 29.9W. During the 5.26sec with compression=on it consumes 2.88Wh for an average power of 31.82W.

To focus even more on the energy specifically consumed by the IOs I can substract the energy  that my laptop consumes when idle, i.e. 22W (see previous article for details). This turns in 2.68Wh for 7m19sec and 1.99Wh for 5.26sec. I end up with the following numbers:

• 0.97Wh for IOs without compression
• 0.89Wh for IOS with compression=on
  

Turning compression on consumes more power but thanks to the extra speed it saves energy with a 1.09x factor - 9%.

On my laptop, compression has a triple positive impact:

• Improves IO performance by a factor of 1.35x
• Saves space by of factor of 1.77x
• Saves energy during IOs by of factor of 1.09x
  

To close this article, let's have a look at the different compression algorithms available with ZFS. For my experiment, I have been using the default by setting compression=on on the ZFS file-system. The man pages tell me that by default ZFS uses the lzjb algorithm "that is optimized for performance while providing decent data compression". Yet, the gzip algorithm is avaiable too, and I can specify the gzip level by using the value "gzip-N where N is an interget from 1 (fastest) to 9 (best compression ratio). Currently, gzip is equilvalent to gzip-6". Let's do a quick test.

First, I delete the data in the file-system where I want to do the test, then I set compression=gzip-1 on it:

# rm /rpool/compressFS/*
# zfs set compresssion=gzip-1 /rpool/compressFS/*

Then, I create a large file - mine is 4.8GBytes - to this file-system and I cp the large file into the same file-system:

# time cp OpenSolaris2009.06.vdi foo.vdi
real    5m4.266s
user    0m0.095s
sys    0m5.903s

# zfs get compressratio rpool/compressFS
NAME              PROPERTY       VALUE  SOURCE
rpool/compressFS  compressratio  2.57x  -

5m42sec for the real-time and 2.57x for the compression ratio. With compression=gzip-9 I get 12m32sec for real-time and 2.82x for the compression ratio.

These values have to be compared with the ones for compression=on (see previous article for the details): 5m26sec for real-time and 1.77x for the compression ratio.

With my laptop, the default compression provides me the best performance, while gzip-1 brings me a much better compression ratio with a minimal impact on perf. Gzip-9 slightly improves the ratio over gzip-1 but with a very negative impact on perf. For sure the differences between these algorithms depend on the disk's speed and on the data being compressed. If I were using and SSD instead of a hard-disk, algorithms with a low compression ratio - hence the onces where CPU is waiting for the disk - would probably see a boost in performance.

The Devoxx conference is around the corner (in just over a week) and will take place in Antwerp, Belgium as every year, only a bit earlier than usual (December was just too close to the Christmas holidays). This week-long conference runs as follows: the first two days are 3-hour sessions to go deeper into the technology (they're called University sessions). The regular conference starts on the third day and offers keynotes and 1-hour sessions. Devoxx also has "Tools in actions" (30-minute), "BOFs", Quickies (15-minute), white boards, and more.

First and foremost, the Wednesday Sun and Oracle keynotes should not be missed (or to be watched later on Parleys) :
• Java, the Platform for the Future - Steve Harris (Oracle)
• Java EE 6 and GlassFish V3: Evolution of a Platform - Roberto Chinnici and Ludo Champenois.

But there's a also long list of GlassFish and Sun-related session for this year's session :

University talks :
• Enhancing the JavaServer Faces 2.0 Component Model - Roger Kitain
• SOA, OpenESB and OpenSSO Programming with Passion - Sang Shin
• The Java EE 6 Platform University - Antonio Goncalves, Alexis MP

Sessions:
• JDK7 Update - Mark Reinhold
• The Java EE 6 Platform - Antonio Goncalves
• Writing Asynchronous Web application (Comet) using the Atmosphere Framework - Jean-Francois Arcand, Paul Sandoz
• Project Coin - Joe Darcy
• Using BTrace and DTrace to Instrument and Analyse Java Applications - Simon Ritter
• Enhancing the JavaServer Faces 2.0 Component Model - Roger Kitain
• Managing GlassFish on OpenSolaris - Simon Ritter
• The Modular Java Platform & Project Jigsaw - Mark Reinhold
• Deep dive on the Java EE 6 platform with GlassFish V3 - Roberto Chinnici, Ludo Champenois

BOFs:
• Grizzzly Servlet Container - Jean-Francois Arcand
• Update JDK 7 - Mark, Alex, and Brian
• The Modular Java Platform & Project JigSaw - Mark Reinhold, Alex Buckley

Quickies:
• Java EE 6 and OSGi. Ludo Champenois

See you there!

A big thank you to everyone who attended the Java Professional Day in Langen. Here's my JavaFX material as promised:

• The presentation in pdf format
• Access to Music Explorer FX a great JavaFX application. Click here to start it
• The path animation demo
• The video cube
• The Java Store demo video (Quicktime) 

Have a look for the JavaFX samples. Lot's of good material!

India as a region that has had a higher penetration rate of mobile phones than the internet & this is certainly an area that is not only growing rapidly but has a dirth of socially impactful, easy to use consumer applications. The "me too" applications with a 100% reliance on ads has failed & continues to draw a lot of suspicion towards monetizing via advertisements. It was an interesting day kick started by Aravind Rao with a coverage of what & how startups & other mobile application developers in India can build & sell mobile applications. There is more coverage provided by Rajiv of WATBlog on Arvind Rao's talk here.

There was a lot covered about AppStores & their benefits to developers across all the discussions & talks. Appstores are certainly here to stay and are a quick application delivery & distribution channel for all developers.

Opinions - Its evident that the VAS opportunities in the country are huge. As ARPU declines, the challenge for operators is to increase revenues by differentiating their offerings and develop alternative revenue streams by offering more value added services to the existing subscribers & finding better means of consumer outreach., to stay in the game.

VAS is the way to go what with one 3G service launched & more looming. Content providers are key here to ensure that they have intelligent means to find out how to provide highly customized, relevant  & contextual content to the large & ever expanding consumer base. While UI design for mobile is important, the reality today is that more content delivered is SMS based & it needs to be provided in a manner so as to support low-end handsets as well. One of the speakers mentioned the ABCDs of most popular content in India as - Astrology, Bollywood, Cricket & Devotion ! This has been the case for the last several years & any associated content still has a large & highly interested consumer base even today. Further, social enterprises are still lacking in the region which is again a greenfield opportunity. With mobiles being ubiquitous as they are today, network services being enhanced by operators, the mobile is certainly the one quick delivery mechanism for app vendors. This is an area certainly worth exploring & investing in. I was surprised that the speakers didn't bring up too much of this aspect.

A very detailed on Mobile VAS in India is available here.

Today, at the PayPal conference, Sun and PayPal announced that PayPal will now be a payment option for the Java Store. To read about the announcement and all the details, check out this article.   Additionally, the Java Store and Warehouse are now live.

 To get the client (i.e. store): http://store.java.com/developerrelease

Developer Portal (i.e. warehouse to upload client): http://java.sun.com/warehouse/ 

Data Deduplication is a big deal, as was shown back in July when EMC spent 2.4B$ to acquire Data Domain. This morning Jeff announced that dedup has been added to ZFS; this has generated quite a bit of buzz in the 'web, although I've yet to see Oracle's stock going up... or Apple changing their mind. Check Jeff's post and comments; it is a nice read. Also read on disk savings at ZFS-Discuss.

Jim Faut and Rick Palkovic have been posting a nice series on how to troubleshoot OpenSSO with Firefox Add-Ons. They just pushed out two more entries in the series, which now includes:

• Part 1: Introduction • Part 2: Single Sign-On and Policy Protection • Part 3: Cross-Domain Single Sign-On • Part 4: Service Provider Initiated Fedlet Single Sign-On • Part 5: Identity Provider Initiated Fedlet SSO Fedlet deployment

These articles are worth a check even if you just want to learn about how OpenSSO works: just follow their diagrams to see the exchange of information between the parties that enable these features.

And, on this topic, you may want to track the participation of the OpenSSO team at next week's Internet Identity WorkShop; see Daniel's note.

And Happy Halloween!

ColdFusion (wikipedia, product) was first released in '95 by Allaire which was later bought by Macromedia in '01 and merged into Adobe in '05. CF was rewritten into Java a while ago, interacts nicely with JavaEE and with Adobe's products and is still quite popular. When we got serious about GlassFish several of us drove down to San Jose to talk with Adobe. Adding a new supported platform is non-trivial for a large vendor; the question is not "does it run?" part but "is it worth setting up my testing and support team?"... which boils down to, "do I see enough traction in my customers?". So, I'm very pleased to point to: ColdFusion 9 supports ... and Sun™ GlassFish. Happy! And Wednesday's news should just help further.

Adding a few other recent GlassFish sightings...

• How to Install JRebel on GlassFish with Eclipse IDE
• JavaMonitor Probe for GlassFish
• Servlet 3.0 support in Maia (IntelliJ IDEA 9) via GlassFish v3
• Installation Guide for JSF+Facelets+RichFaces (Technical BrainWave)

Also, check out: Sun's Technology powers Verizon Developer Community.  We really need to get back to posting adoption stories - there have been quite a number of great ones in the last few months.

GlassFish v2.1.1 is out (Sun Distro, Community Distro). GFv2.1.1 is the foundation for SailFin v2 and includes refinements on Replication and Failure detection plus many (>200) bug fixes and other improvements. See Shreedhar' s Overview, Kevin's post, the Wiki page and PR @Oracle OpenWorld. GFv2.1.1 also includes OpenMQ 4.4, Grizzly 1.0.30 (changes), Jersey 1.0.3 (changes), Shoal 1.1 (changes) and JSF 1.2_13. The bulk of the changes are from the GF repository (changes).

The commercial offering is via the GlassFish Portfolio. Note that GFv2.1.1 is also a patch for earlier releases (GFv2.1, itself a patch for GFv2U2) but the patch has not yet published at SunSolve. I'll post an entry at GlassFishForBusiness when it becomes available.

James reports on the latest news on JavaCard 3.  JavaCard (wikipedia, spec site) is what we all have in our pockets (ATM cards) and/or our phones (SIM cards). The JavaCard 3 comes in two Editions: Classic (for SIM/ATMs) and Connected. The new kid, Connected, supports most of the JDK6 VM as well as Servlet 2.5, extended and classic Applets, HTTP and HTTPS, etc. The target of JavaCard3 Connected includes secure USB tokens and personal DBs, Embedded Servers, WebDAV-Compliant Thumb Drives, etc.

JavaCard Connected seems it may deliver on the promise of "connected" Java devices everywhere; we will see how it gets adopted. There is a new project at Kenai focused on learning about JavaCard Connected. The project includes the NetBeans Plugin (see sneak preview).

Today was the release of SailFin v2 (download, home, wiki) and its companion Sun GlassFish Communications Server 2.0 (download, home). SailFin v2 is a big release; it leverages GlassFish v2.1.1 (more tomorrow) and adds a number of features including high availability, rolling upgrade, flexible network topology, better overload protection, Diameter support, improved diagnosability, Java based DCR files for the load balancer, and more. I can't cover SailFin v2 properly but I'll collect some of the relevant links so you can follow them up.  Start with Binod's overview: SailFin v2 Released! and move from there:

Some of the major changes are:

• A new replica selection algorithm that improves scalability by easing load distribution after failure.
• Enhancements to the Converged Load Balancer to support writing DCR rules in Java.
• The ability to do rolling upgrades leverages session replication; see this example.
• Improved use of multihoming features for more flexible deployments. See Ramesh's writeup.
• Diameter support in Sun's for-fee AddOn; see details in Venu's many posts, including [1] and [2].
• Improved load protection.  See Robert's one pager and Sankar's work on JMX Notification.

As part of the release, the team has posted a number of new entries, including:

• Next is what? (Prasad)
• Subscribing to user state using Sun Diameter Sh API (Venu)
• Setup and Configure Diameter on SailFin (Naman Mehta)
• Diameter Administration on SailFIn 2.0 (Yamini KB)
• SailFin v2 Released (Srikanth Anandal)
• CLB Enhancements in SailFin 2.0 (Kshitiz Saxena)
• Overload Protection in SailFin: What's new in 2.0 (Rampsarathy)
• Tips and Tricks of SailFin HA (Bhavani Shankar)
• Multihoming in SailFin. (Ramesh)

Older posts worth checking include:

• TA entries tagged SailFin,
• Prasad's earlier roundup: "SailFin 2.0 is Around the Corner!"
• Sreeram's notes on SailFin amazon images.
• How to write Conference App in 15 lines of nicely formatted code!

Finally, a list for PR/Press reports:

• Sun Communications Server Speaks VoIP (The H Open Source)
• Barron's, BusinesWire.

Note - GlassFish v2.1.1 is also available from Sun's Download Center and from the Community Site. More on that release tomorrow. And the OpenMQ 4.4 (and 4.4.1 RC1) are available from here.