In September, 2009, we released version 2.0 of the Sun Machine Learning Engine (SMILE), Sun's own advertising-oriented predictive analytics system. The primary customer facing manifestation of v2.0 is the yellow-bordered SMILE Recommendations "Carousel," which you see on many sun.com web pages:

As it's been more than four months since its release, I'll present some of the measured results. Overall, we've been very pleased with the value delivered by the carousel. (There have been some negative reactions, too, but I think that's to be expected when introducing highly-visible new advertising into the mix. More on that later...)

Bottom line, the carousel has exceeded our expectations and far outperformed the small ads we displayed using SMILE v1. Ads (also referred to as "recommendations") in the carousel were clicked by our visitors as often as one million times per week, yielding an average click through rate of over 3.6% (and the rate was much higher even on some sections of the site as opposed to others, depending on the audience). If you're familiar with online display advertising, you'll know that's a pretty phenomenal CTR, and we attribute it to the effectiveness of the SMILE analytics and recommendation engine, strong inventory of content and offers, and highly visible placement.

Clicking on an ad, though, is just the start, so we also measure what happens next, our so-called "key success metrics." These include:

• Downloads Initiated
  Many ads point to either the Sun Download Center (encouraging visitors to get our free software) or specific software downloads, and SMILE drives on average 14-16,000 downloads per week. This is very beneficial, as getting our software in customers' hands is often their first real engagement with Sun.
• Offers Obtained
  We provide many white papers that require simple registration or login to obtain, and the carousel often advertises and links to these offers (here's an example offer). They provide valuable contacts and leads to our Sales team, and SMILE drives on average 6-7,000 of these new contacts per week.
• TeleWeb Success
  Here we measure how many visitors initiate a contact using the "TeleWeb widget" (pictured at right) after clicking a SMILE recommendation. Each such contact has a significant potential sales value (that we've measured accurately over time), so it's significant that SMILE delivers 100 or so contacts each week.
• Sun Startup Essentials (SSE) Applications Submitted
  The SSE program is a win-win for Sun and startup companies, providing great value to each over the long run. SSE created a number of SMILE ads, and they perform very well, driving up to 25% of the weekly applications received into the program.

We measure many other variables and events, and in fact, thanks to recent enhancements, can now do closed-loop reporting with some of our Tele-Sales teams. This takes reporting one step further, providing the actual potential dollar value driven by SMILE. Here's how it works: When contacting a lead generated through a SMILE click, sales reps enter into their CRM system an estimate as to the potential value of the lead. This doesn't include all leads generated, as they go to different teams, and not all of them have enabled closed-loop reporting.  There is also a time lag between lead generation, dissemination, contact, and possible assignment of marketing pipeline value into the system. Even with those caveats, SMILE generated over $2,500,000 in potential leads value in December, 2009, alone! This is a great example of our push towards measurable, "deterministic" marketing and how we can realistically start to calculate ROI on this project.

Our real-world experience also comes with customer feedback, and we see room for improvement in the following areas:

• First, do a better job of surfacing and explaining how the carousel and the recommendations work. The info has always been there but was "buried" on the All Recommendations Page. We would like to add an "About Recommendations" link to the bottom border of the carousel so that curious and/or concerned (with privacy) visitors can easily learn more about the program, how it works, and how we handle privacy related matters. The link would go to the bottom of the All Recommendations page where we've recently added a new section on recommendations, privacy, and program FAQs.
• When a visitor closes the carousel, it stays closed for the duration of the browser session. But for users who visit often, this was inadequate. We would like to implement a different solution that keeps the carousel closed longer for users who don't wish to view it.
• Due to an initial basic mapping system between images, products, and ads, we sometimes show duplicate images and/or lack image variety. We've been adding new images to the system to address this.
• There are a few more simple enhancements that will also increase variety in the carousel, such as not showing slight ad variations for the same product or offer at the same time, and eliminating ads that link to the page the user is already on.
• Lastly, on the back-end, we continue to refine and enhance the recommendation engine and methodologies with a goal of always increasing the relevancy of the ads to our visitors.

Will these enhancements see the light of day? Some decisions are pending the closure of the Oracle acquisition, so time will tell.

Regardless of what happens, though, I hope this information helps convey the strong results the program has produced, the success of our predictive technology, how it can be further improved, and the promise such systems hold for the future.

Thanks to Frost & Sullivan for inviting me to their Web Experience Excellence 2009 Executive Congress held in downtown San Francisco on December 2, 2009. The specific topic was "Driving Strategic Online Advantage & Advocacy: Are your customers your evangelists?" and it was highly focused on how enterprises can and should use social media marketing. It pretty well covered the gamut from corporate blogs, wikis, and forums to Facebook and Twitter.

I'll give a quick run down on some of the most cogent "sound bites" and speakers of the day, but first a few observations. There was a lot of talk about "strategy" and is there such a thing as a "social media strategy." Some felt it was essential to develop one if not done already. Others felt that social media was just a part of the overall marketing/product strategy and not as such an end in itself. When considering the many examples presented of how social marketing is handled, it seemed the "strategy" answer ran all the way from none to formal. In some cases, it was (not surprisingly) more of a grass roots movement within the company, starting more with individual usage of social media, then moving up the "food chain" based on individual initiative and success. It's only recently that some companies have much more formally institutionalized the process, with dedicated social marketeers, teams, road maps, and indeed strategy. 

In all the case studies presented, there was almost no talk of "failures," as most of these initiatives, from grass roots to highly planned, have yielded positive results. In fact, this is such a hot and fruitful area that you may need to "just do it," even if the exec backing isn't all the way there yet. How and what to do were the focus of the presenters.

Alexander Michael, VP, Information & Communication Technology at Frost & Sullivan was our moderator and kicked off the first talk along with James Latham, Sr. VP, Strategic Marketing, at Open Text (which co-sponsored the event). Their talk on "Emerging Customer Expectations: Hyper-Technology and the Evolving Online Experience" set the stage for the day with some impressive stats on the growth of social media and its growing importance in the enterprise. 

Playing on the saying, "It's the economy stupid," they proposed the Web 2.0 version, "It's the people driven economy, stupid." Other key bullets from their talk:

• Social media is not a fad. 
• Don't communicate to GenY, communicate with.
• Realize the new generation on the web has no loyalty. You must try to earn it with an engaging experience and seamless integration of social technologies. Three minutes is all you have...
• You must encourage and not fear customer feedback. It's about Community, not Negativity. Most customer reviews are generally positive.
• Don't try to fool people -- putting a web veneer over manual back-end processes won't work any longer.

Jeben Berg, YouTube Marketing Programs, Chief Innovationist, gave an entertaining and engaging talk, "Online on the Go: The Mobile Web and Impact of Video." He illuminated some clever YouTube marketing campaigns and offered great advice on how to succeed in that arena. I was impressed with his enthusiasm and domain expertise -- if you're developing a video campaign, you'll want to talk with Jeben. A few highlights:

• Be who you are and be thick-skinned as well. YouTube won't take down parody or negative videos about your company or product -- they are a platform provider, not a content provider. The best response to a negative is your own video reply, done quickly.
• Humor and irony rule. Emotion and competition resonate. "Low fidelity, high concept" works.
• There's an audience for everything.
• It's not about technology, it's about trust.
• Videos should be self-contained -- you don't know where or when users are coming from.
• Some examples:
• Clever, very subtle advertising: Guy back flips into jeans (Levis)
• Just plain cool: MUTO a wall-painted animation by BLU
• Viral phenom with the kids: Fred (warning: could make you instantly insane if over 12 years old...)

Mark Yolton, Sr. VP, SAP Community Network, and Salim Ali, VP, Enterprise Solutions and Community Marketing at SAP, discussed, "Utilizing Social Media: Harnessing the Power of Online Communities for Loyalty and Advocacy." They talked about the evolution of communities at SAP and how they're used to Connect -> Collaborate -> Co-Innovate. They also talked about how to measure success, an important topic since typical "hard ROI" is often hard, if not impossible, to calculate with social programs. Their measurements center around number of members, traffic, contributors, and momentum. One interesting concept they raised was around segmentation, which is typically based on who you are. In the community space, what you're talking about is really important too -- segmentation by conversation.

Angela LoSasso, Global Social Media Strategy & Programs, HP, was up next for a discussion on "GenY and Beyond: Customer Experience in a Web 2.0 World." She talked about distinct benefits, such as how social can drive "Google juice" (think link relevance) and build brand relationships early on with younger (potential) customers. She was a proponent of crafting a social strategy, cautioning to first learn how your customers use social media before defining your approach. Another good point was not to confuse your strategy with the technologies and tools that enable it. Two other take-aways:

1. Social and mobile are now one. (This was a recurring theme -- do not overlook your mobile customers!)
2. Twitter can serve as an "idea factory."

James Latham spoke again next, talking about "Best Practices Live" and the Seven Essentials:

1. Strategy with measurable goals
2. Use all available inbound tools
3. Content is king
4. Be relevant
5. Social media is here to stay
6. Actionable analytics are the only metrics that matter
7. Leverage your investments

James then introduced Christer Ljungdahl, Director, Web and Direct Marketing at National Instruments, who showed the "essentials" in practice at NI. Similar to SAP, there was a lot of focus on customer forums as a key interaction place for technically-oriented enterprises and their customers. There's certainly a cost avoidance benefit when customers provide each other tech support, and it's often faster and better than sitting on hold for tech support. But really these forums build community, and as Chris noted, "It's more believable when they say it" (think  product recommendations and reviews). His approach to building evangelists: Enable -> Share -> Listen -> Respond -> Recognize. One non-employee on their forum recently put up his 20,000th post! Forum managers observe a 90/9/1 rule: 1% of visitors are heavy contributors, 9% participate, and 90% visit.

It wouldn't be fair nor accurate to say they "saved the best for last," but Victor Cho, VP & GM, Consumer Internet & Software Services, Eastman Kodak, was an excellent presenter with clever, thought-provoking visuals. He engaged the audience in his talk on "Aligning Your Online Team with Corporate Objectives," and more specifically, "Eight ways to get your online team to truly deliver:"

1. What do you measure?
   Typically the customer (like NetPromoter scores), shareholder (profit, market share, etc.) and employee satisfaction. To that, he added "Competitive Advantage." For example, note how Google focuses (some would say obsesses) on speed as an advantage.
2. "Walls not screens" -- there's lots to measure, and focusing on one thing may miss the big picture. Perhaps your conversion rates are up, but sales/traffic overall are down. Profit might be up, but is it because you cut costs rather than increased the top line and margins?
3. Measuring the customer -- figure out what causes (and how to measure) "wow" vs. "pain". Consider process vs. speed. Re-engineer process around newer/better customer experiences plus the need for speed.
4. Conflicting speeds -- how often do you measure, yearly, monthly, weekly, daily, hourly, realtime? You need to measure different things at different intervals. Weekly feels right for many web metrics.
5. Don't believe everything you see. Metrics don't always represent "the truth." There is still a very important place for judgment, testing, and experience.
6. Gain from conflict, such as revenue vs. customer experience. He showed a formula, "$$$ = QV" where Q = quality and V = velocity (not value). Information spreads so quickly now that velocity is essential.
7. Channel is another source of conflict -- you must balance and resolve channel conflict.
8. Velocity of change -- when's the right time to push changes into the system? How do you balance quality versus speed? Will the change improve task completion? What changes will push your key metrics higher? You might think you're at the zenith, but then again, you might not be seeing the entire universe. No change, no gain.

So, with a final thanks to Frost & Sullivan for the nice drinks and hors d'oeuvre at the closing reception, that's a wrap.

Recently, we released a new feature to the download platform that powers the Sun Download Center.  Internally, we refer to it as the Advanced Download Widget (ADW).  Essentially, it's a Web component that can be deployed on any Sun-branded Website, to deliver an integrated and streamlined download experience.  Working with Lifecycle Marketing, we have also integrated the ability to present the user with a free offer (e.g. whitepapers, training, etc.) that complements the download.  It's completely optional, but to receive the free offer, the user will need to login with a Sun Online Account or create a new one.  To see it in action, checkout the Java ME SDK 3.0 download page.  Although the project was generally successful, there were some lessons learned (in terms of went well and what can be improved from my own personal perspective) that I would like to share via this blog for future reference.

When developing new Web functionality that relies heavily on Web browser technology, it's important to understand your users.  Do they largely run on a single Web browser / platform combination (Intranet apps), or do they span the gamut in browsers and platforms used?  The answer may greatly affect your project plan and testing strategy, so find out before you start on your project.  Sites such as Wikipedia publish aggregated Web browser usage stats for the Internet, but it is better to take your own measurements if possible.  Our sampling of a very popular Java download yielded slightly different distribution with 48.6% running Internet Explorer and 40.8% running Firefox.  The data helped shape our testing strategy.

Given the constant evolution of the Web browsers, it's not always practical to maintain backward compatibility to outdated Web browsers; however, forward compatibility for new releases should definitely be a priority.  Internally, you want to establish guidelines on the Web browser makes and versions, as well as the browser platforms that you can support.  This way, you can provide Engineering and QA with clear expectations on the testing scope and staffing needs.   It's also good to have designated personnel who keep tracks of the product roadmap for key Web browsers.  During the development phase for the ADW, new version of Firefox (3.5) and Safari (4.0) were released, but they were not on our radar.  We later uncovered some minor incompatibilities with these Web browsers during the testing phase that prompted additional round of testing and contributed to some avoidable schedule delays.

One key aspect of project management is accurate planning of the time and resource it takes to complete the project.  While the Development and QA phase are largely determined by Engineering's estimates, the business generally drives User Acceptance Testing (UAT) and therefore estimate the resources and duration required for UAT.  Because the project scope varies from release to release, successful UAT planning requires a good blend of gut feel (art) and common sense (science).  By applying past experience and intimate knowledge of the system to the bug fixes and enhancements in scope, you can devise a rough approach to the UAT test plan and test cases.  From the estimated time for completing each test case and the availability of UAT testers, you can then derive the duration required to complete 1 round of user testing.  When testing new features that relies heavily on Web browser technology, be sure to add extra time and/or testers for targeted cross browser/platform testing.  Finally, allow time for bug fixes and at least a second round of user testing to make UAT a success.

Finally, while the project life cycle ends when the release goes live, the product life cycle continues on.  By product, I'm referring to the system or the download platform in this case.  Although it's possible to roll out new features that meet the user's needs on day 1, quite often the road to nirvana involve a couple design iterations.  To avoid the design-in-a-vacuum pitfalls, it's essential that you have ways to collect feedback and gain insights into the user's real world interactions with your system.  In our case, we chose to conduct a usability study to gather feedback from a diverse group of users whose experience with Sun ranges from none to developers and Sun customers who are quite familiar with our Websites.  Although the usability study was insightful, we plan to integrate Omniture into future releases so we can measure and assess the usability of the ADW across the entire user base.  Meanwhile, we have other ways (see my blog on The Voice of the Customer) to collect and act on customer feedback as well.

Thanks to my colleague Paul Strupp for bringing a recent New York Times article to my attention: "Two-Thirds of Americans Object to Online Tracking." This is the conclusion of a recent survey conducted by professors at the University of Pennsylvania and the University of California, Berkeley. As I'm currently the Project Manager for an online marketing system that leverages online tracking, this was obviously of interest to me and our team. The title of this post does a good job of summarizing the general negative attitude of those surveyed, and I won't rehash what's in the article, trusting you'll give it a read if interested. After reading the article as well as the full study, here are some of my thoughts.

My first reaction is similar to this quote in the article from Stuart P. Ingis, a partner at the law firm Venable who represents the industry trade groups' self-regulation coalition: '"Just because many Americans are not in favor of something does not mean it should be banned," he said, citing negative feelings about taxes.' 

I think the tax analogy might be a tad extreme, but you could survey Americans and find a zillion things they don't like but that no one has any intention of banning. Ask me how I like the fact that TV ads are now inserted in the shows themselves (instead of just in between the action, where you can mute or fast-forward through them), and I could give a pretty good rant about how much I dislike that practice (OK, it's a personal pet peeve). But it's part of the price we pay for free TV, and I don't see any calls to make it illegal. I acknowledge that personal privacy and TV commercials aren't necessarily of equal weight in the "big scheme of things," but hopefully you get the idea, so I'll leave it at that.

Bottom line, businesses derive value from these systems, and at Sun, we see measurable positive results from our use of predictive analytics, many of which rely upon online behavior data. (The results from our recent release of Sun Machine Learning Engine v2.0 are even more positive, but I'll cover that in a subsequent post.) While we do aggregate data from Sun-owned domains, we don't aggregate from non-Sun domains like many ad networks do, and users in the study objected slightly less at least when it was all within the same company. 

We also have direct feedback from Sun customers who participated in a recent usability study about our project, and their attitude was quite different. They said they had an expectation that a large sophisticated enterprise web site would track their online visit, and if we used the data to provide them recommendations that were accurate and helpful, they had no issue with it. (The nature of our business was important too -- they said they'd feel differently if we were their financial institution, for example.) They noted how large our site is and that it can be tricky to navigate -- if we can help them find valued information more quickly, they were actually quite supportive. Granted, this was a very small sample and not scientific like the survey, but I just wanted to point out that some in Sun's audience have a different attitude.

That said, I'm as paranoid about my personal information and privacy as any one -- I always disallow third party cookies, opt-out of advertising networks cookies (as best I can), shred everything, etc. So I totally get it. Further, I agree the onus is on the businesses using online tracking to do a much better job of assuaging concerns and communicating. I suggest three best practices:

First, transparency is critical, and we must be up-front about what we do. Sun has an extensive Privacy Policy that I feel is understandable, honest, and forthcoming. We went a step further as well to add specifics about the online tracking we've implemented recently, which you can see on the new "All Recommendations" page:

It's tricky to try to explain this adequately in as few words as possible (on the assumption that users typically don't want to read that much). Hopefully in this short blurb, we communicate at a high level the way in which we've implemented our recommendations in relation to tracking.

Second, users should own their own data and be given control over it. I can't say we've totally got this right yet, though we do allow a general cookie opt-out which will prevent us from tracking anonymous online behavior. I think technology has a ways to go here, as making your "average" user manage their cookies at this level isn't a great solution. There's room for improvement and advancement in this whole area of customer data management and empowering them to manage it painlessly, though I think Sun is doing pretty well in comparison to many other companies.

Lastly, like any change you want folks to accept, you must answer "What's In It For Me?" It's a negotiation -- we're asking users to let us track their online behavior in order to make recommendations that we believe will benefit them (and us of course), potentially in a number of ways:

• Fast, (usually) free access to relevant information, including informative white papers and no-cost software downloads the user might not otherwise locate
• More direct acccess to potentially valuable promotions, specials, and offers 
• Aggregation of all the info into a single, simple user interface

To explain benefits explicilty would require yet more words, and that's a bit of a conundrum for the web experience. Hopefully the experience itself conveighs the benefits adequately (and customers can always read my blog to get the full scoop ;) .

Again, relating to this personally, I know that by disabling my ad network cookies, I might miss some targeted ads that might resonate with me, but that's my choice. On the other hand, I wouldn't want to deny Netflix the opportunity to recommend movies to me, so I explicitly tell them what I like and don't like and don't worry that they know about every movie I watch. Similarly, I don't try to turn off product recommendations on Amazon that I often find helpful. This is a trade-off that works for me. I'm hopeful that as we get further down this path, we can do a better job all over the Internet of transparency, data management, providing customer data control, and communicating (and delivering) benefits. If we do, maybe someday in the not too distant future, only "One-Third of Americans" will object!

We can thank my wife for this posting...

We recently sent our son off to college with his new laptop computer with Windows Vista on it. (After having worked with Vista for some time now, I definitely understand why people dislike it but also find that it works fine for most everyday computing needs.) I don't think he realized the value of living with a "systems administrator" (that's me, tamer of the home computers) so I sat him down, and we had a long discussion about how he darned well better back up his computer and take care of it. In other words, don't come crying to me if a virus eats your term paper 2 days before it's due and you don't have a backup!

To try to make this as "easy" for him as possible, I wrote up the method I've developed over many years for maintaining my Windows computers. This is where my wife comes in -- she said I should share this with others since it seems to be pretty good advice, so here you go.

There are weekly tasks and monthly. This is a reasonable amount of effort. If you really really want to be safe, replace "weekly" with "daily" and "monthly" with "weekly." (And no, I don't really expect my son to do all of this, but you can always hope. Bottom line was I said to him at the very least promise me you will back up your entire computer once a month, promptly install Windows updates, and make sure your anti-virus is current and working.)

By the way, the following works almost identically on Windows XP and is just as applicable to that operating system.

Weekly

• Use an account with administrator privileges for the following (but always do your everyday work and web surfing in a non-admin account). 
• Start with a reboot and all programs closed.
• Make sure Windows is up-to-date and that there are no outstanding Windows Security Updates. Install all updates if needed.
• Temporarily disable your anti-virus program before running Windows Update -- AV programs have been known to interfere with some updates. 

Why waste back-up disk space on garbage? Clean up the system first:

• Use the built-in Disk Cleanup program to remove large, space-wasting old System Restore files.
• Double-click (open) "Computer" (or "My Computer").
• Right click "Local Disk C:" and then select "Properties" at bottom of pop-up menu.
• Click "Disk Cleanup" button.
• Select "Files from all users on this computer" & wait for scan to complete.
• Click "More Options" tab on Disk Cleanup window that comes up.
• Under "System Restore and Shadow Copies", click "Clean up..." button and wait a little while.
• Click "OK", then "Delete Files" on pop-up box.
• Close the open windows when done.
• Run CCleaner to clean up a bunch more garbage.
• Note this program may erase all your cookies, and you probably don't want that. (It's fine to keep cookies for sites that you use regularly, so you don't have to login every time for example).
• If you don't want to erase all your cookies, click Options -> Cookies, then move selected cookies to the Cookies to Keep list.
  

Now that you've cleaned up the disk, the next set of steps will run a bit faster.

• Make sure anti-virus files are updated & current (we're using Symantec, so run their Live Update program if needed).
• Run a full anti-virus scan (allow 30-60 minutes; run "Quick Scan" if in a hurry).
• Run a general utility that cleans up the registry and may do other housekeeping. We're using the free Glary Utilities, which seems to do a fine job.
• With Glary, run "Scan for Issues" first, then click "Repair Problems" if any are found.
  

Give your hard drive a little TLC -- it can go a long way to preserving the drive and your data over the long haul.

• Run the command line utility chkdsk to ensure the drive is healthy and fix any problems:
• Open a command line prompt as an administrator:
  Start -> Programs -> Accessories -> right-click "Command Prompt" and select "Run as administrator" from pop-up box.
• At the command prompt, type:    chkdsk c: /f
• Enter "Y" to run next time system restarts.
• If you have more than one partition, repeat for each partition.
• Restart the computer so that chkdsk runs. 
• One odd thing I've noticed on Vista is you have to be really patient after checkdisk finishes and before the computer completes rebooting. I've seen it just sit there for a long time like nothing's happening, but just keep waiting, and the system will restart.
  

Monthly

I'd recommend doing these tasks the first of each month. (Skip to next section if not monthly.)

• Check all (or at least most) of the software on your system for security issues and updates. It's much more than Windows nowadays that may be an attack vector. For this, I run Secunia PSI.
• Click the "Scan" tab and wait for it to finish scanning.
• "View Insecure Programs" if found.
• Replace insecure programs with the newest versions.
• Defragment your hard drive -- it may run a little faster afterwards. Keeping with the free utility theme, we've been using Smart Defrag.
• Check the box to defrag the C: drive.
• There's a box next to the Start button that might say "Defrag Only". Change it to "Fast Optimize".
• This usually takes a while! Give it 1-2 hours or overnight maybe.
• You can use "Defrag Only" if in a hurry -- it only takes a few minutes but it's not as thorough.
  

Weekly, continued...

And now for the most important part, back up your system! Actually, it's not really a back-up -- we're going to make an image. This allows you to restore the entire hard drive, including all you data, exactly as it was when you created the image. If you just back up data and your hard drive dies or gets infected, you have to reinstall all your software, retweak everything, and then restore your data. That's why I prefer images -- in one step, you can return the entire computer to the exact state it was in when you created the image.

• Restart the system one more time. (This might be a bit of overkill, but I want my image to be based on as clean a state as possible.)
• Temporarily disable the anti-virus -- the imaging may go a bit faster.
• Now make an image of the entire hard drive. There are various programs you can use, but I am a big fan of Acronis True Image and highly recommend it. Granted this one's not free, but it's a small investment for the value and peace of mind.
• Start Acronis True Image and make an image of your C: drive to the external hard drive. (You can use Acronis to back up to a network or DVDs, but the easiest thing is to use an external hard drive I think.)
• Make a new, complete full image at the start of each month.
• Make an incremental image on top of the full image each week, until the next month.
• As the external drive fills up, make room by deleting the oldest images.
• We're just about done. At this point, don't forget to re-enable your anti-virus.
• As we're now highly reliant on the external drive, I would run chkdsk on it as well to ensure it's healthy and fix any problems.
• Open a command line prompt as an administrator:
  Start -> Programs -> Accessories -> right-click "Command Prompt" and select "Run as administrator" from pop-up box.
• At the prompt, type:    chkdsk g: /f
  (Note: Replace "g:" if needed with the correct letter for your external drive.)
• Properly shut down the external drive, then unplug it and turn it off.
  

And now you're done.

So, is this process really worth the effort? IMHO, absolutely!

• Ever have friends tell you how they have to reinstall Windows (sometimes often) because of all the problems they have? I worked on a Windows XP system every day for 6 years and never had Windows crash nor did I reinstall it once. I believe maintenance is a key to this.
• Ironically, right after lecturing my son about all this, my hard drive on my new desktop (less than a year old) completely and totally died. (Seems Seagate made some faulty drives, and I was unlucky enough to get one.) The drive worked fine on Sunday when I did my weekly maintenance and imaging. I turned the machine on Monday morning and got the dreaded "Hard drive not found" message. I replaced the drive, restored my image, and it was as if the drive had never died. This is not the first time (and probably not the last) that Acronis has totally come to my rescue. It is so worth it. Please don't wait for this to happen to you to backup your system!

Of course there are many other utilities out there, and everyone has their favorites, and they may have different pros and cons. I'm happy to say I've had good results with the ones I've pointed out, but, as the saying goes, "your mileage may vary." In any case, I hope you find this advice helpful -- feel free to share it with your son or daughter before he or she heads out to college too!

I wrote an overview earlier this month about Predictive Analytics (PA), what it is and what sorts of benefits it offers. I've also written about the Sun Machine Learning Engine (SMILE) Project, putting PA to work on Sun's web sites. Today marks the end of SMILE Phase 1, as we are in the midst of releasing SMILE v2.0. You'll see the results on www.sun.com over the course of the next month, as we gradually enable more and more of the site with our new "Recommendations for you" carousel. Here's what you'll be seeing soon!

So I thought today would be a good time to touch on the results from SMILE 1.0. The initial release was simply about serving small text-only ads in the right hand column on many sun.com sites. Here's an example of a SMILE-served ad (outlined in red):

To evaluate results, we calculated standard metrics, such as impressions (number of times the ad was displayed), clicks (number of times the "call to action" link in the ad was clicked), and CTR (click-through rate -- number of clicks divided by number of impressions, as a percentage). I'm not able to share the detailed CTRs at this time, but suffice it to say they're low. That's not unexpected -- these are small ads, easily overlooked or ignored, and you shouldn't expect a lot of clicks on this type of banner ad. 

However, we also calculated Uplift, and that's where we looked to measure the power of our analytics. We did not have recommended ads for all customers, just return visitors with anonymous cookies that we recognized from earlier visits. Thus, we served many default ads, as well as many recommended ads. By comparing the CTRs of both, we can explicitly measure the influence of our predictive system, measured as "uplift." Here's a simple example:

• On a web web page, we show 100 SMILE-recommended ads that get 15 clicks, for a 15% CTR.
• On the same page, we show 100 default ads (same size, same location, just not personally targeted), and they get 10 clicks, for a 10% CTR.
• The SMILE Uplift in this case is 50% ((15-10)/10 * 100).

We carefully tracked SMILE Uplift for the last five months, and we saw an average uplift of 58.3%. As we serve millions of ad impressions, that translates into 1000's of additional clicks generated by our PA system. The ads often point to downloads or white paper offers that customers sign in to get, and thus we collect 1000's more contacts and what they're interested in, which we can then (hopefully) turn into qualified leads and ultimately new customers. So we can see a definite ROI for this effort. And keep in mind this was "version 1" of the analytics, which we're continuously refining, enhancing, and developing -- we expect ongoing improvements in future results.

Actual weekly Uplift gyrated pretty wildly -- here's a summary chart:

You can see general improvement over time as we improved the algorithms, steadying for the most part in the 40-80% range. In the last week, we released SMILE ads on the Sun Download Center, which had (as you can see) an interesting impact on Uplift! SDLC gets a huge volume of visitors, and most users are there to download and nothing else. We also found a large proportion of users there for whom we did not have recommendations (either because they were new or they'd deleted their Sun cookies). The result was a pretty big dip in CTR for the default ads, while we held steady on the recommended ads, thus the skyrocketing Uplift score the last week.

With the release of SMILE 2.0, we're completely changing how we do our measurements (it's a long story), so we'll be tweaking our weekly measurement system and reporting. We'll have new functionality and new measuring capabilities, and I'm looking forward to seeing the results from our newest release.

As I hope these numbers portray, we've demonstrated solid benefit to our emerging PA technology. It's a great start, but there's still a lot of upside potential remaining -- we're optimistic of delivering even more dramatic results in the future. 

As I've written about previously, I'm currently the Project Manager for the Sun Machine Learning Engine (SMILE) project, based on predictive analytics (PA) technology we're developing in-house. While I have a lot of experience building and managing complex web systems such as this, I haven't worked with PA technology before. I set out to learn more about it, for two reasons:

1. Since I'm the PM for this project, it's generally a good idea to know what I'm doing and talking about!
2. ROI is very important, both for this project and for ensuring the ongoing application of PA technology in general at Sun. This matters, of course, to Sun's management, and as you might imagine, it can't hurt to convey these benefits to our soon-to-be new owners as well. 

So, I set out to learn more about PA, what it is, and what benefits it offers. In this post, I want to share and consolidate some of my findings -- hopefully this will be helpful to others who are considering or starting similar projects.

Now, before I get much further, credit where credit is due. The title of this post, "If You Predict It, You Own It", is a tag line I like, taken directly from Eric Siegel's Prediction Impact site. I recommend this site for an intro to the subject, as it offers many helpful articles as well as resources, such as the Predictive Analytics World conferences and training programs. 

So what is PA?

• Wikipedia has a good article on PA, starting with a concise summary definition: "Predictive analytics encompasses a variety of techniques from statistics and data mining that analyze current and historical data to make predictions about future events." The Applications section offers an excellent overview of the many uses for this technology. Currently, we're using SMILE primarily for  Cross-sell and Direct Marketing. The last part of the article focuses on Statistical Techniques, an area I'll leave to the statisticians and engineers building SMILE.
• Back to Prediction Impact, scroll down the home page and you'll find an informative section called "Prediction Impact Publications and White Papers." To get started, I recommend, "Predictive Analytics with Data Mining: How It Works."

That will give you a good, quick intro to PA. What about the results? What's out there we can leverage to help sell such projects within our organizations? I did some of my own research into this and was also fortunate to have assistance from Sun's Digital Libraries & Research staff in locating a few additional publications. Here are some representative quotes/stats/images that make strong "sound bytes" in support of PA!

Optimizing Customer Retention Programs
by Suresh Vittal with Christine Spivey Overby and Emily Bowen, Forrester
October, 2008

• "Marketers have long relied on analytical techniques to identify and reduce customer churn. For instance, segmentation models help marketers to better profile customers and understand behavior, while cross-sell and upsell modeling deepens relationships and creates barriers to exit."
• "Marketers who target all types of respondents, not just the positives, risk wasting valuable resources on indifferent customers or at worst even triggering churn. This is especially critical in this climate of pressure upon marketing spend."
• "Telenor found that by only targeting persuadables, it was able to reduce overall churn by 1.8%. A more telling statistic: These improvements were driven by only targeting 60% of the potential churners. The benefits of targeting smaller groups is clear — cost savings achieved from fewer contacts by telemarketing and lowering of customer fatigue through selective contacts." 
• "The combination of increased retention rates and lower cost means Telenor will realize an 11-fold increase in uplift campaign ROI when compared with existing programs."

• "While the Internet and new technologies aren’t crystal balls, the sheer wealth of information that can be gleaned about today’s customers—and then applied toward anticipated future behaviors—is staggering. Failure to take this information into account is like leaving money on the table, or worse. You could simply hand it over to your competitors....Today’s smart companies use data, and the insight gained from it, to predict customer behavior."
• "In one example, American Airlines used predictive analytics to better understand the relationship between various customer segments and differential flight patterns. They achieved sky-high ROI results of nearly 1,200 percent in a period of two months."
• "IDC report studied dozens of companies and hundreds of predictive analytics projects. It found that the median ROI for the projects that incorporated predictive technologies was 145 percent, compared with a median ROI of 89 percent for those projects that employed only traditional analytics."
• Nice summary chart from this article:

"Mob Marketing" Webinar and Presentation
Suresh Vittal, Principal Analyst, Forrester Research
Jack Jia, CEO, Baynote
December, 2008

• "Relevant and personalized interactions are critical for enhancing customer experience."
• Baynote quoted the following benefits for their recommendation technology: 
• 40% Lead Lift
• 20% Net Revenue Lift (40% profit lift)
• 400% Engagement Lift
• 1000% Search Lift 

A vibrant and active amount of commercial activity also lends credence to the power and value of PA, and here's info on some PA providers:

• I ran into the aforementioned Baynote at a trade show a few years ago and was intrigued with their technology.
• We were recently approached by Angoss, offering "Proven Results with On-Demand Customer Analytics." In perusing their site, I found a nice collection of "success stories."
• Newcomer Demdex sounded interesting, based on a recent TechCrunch article.
• Iterating.com has a long list of PA products and providers.

And finally, just last week IBM bought perhaps the "granddaddy" of enterprise PA providers, SPSS, for $1.2 billion in cash -- a very serious endorsement of the power and value of PA! 

As noted, we are taking a DYI approach here, and you might be wondering about our results so far. I'll let you digest this info first, then follow up soon with a post on how SMILE is performing...

While my job responsibilities remain essentially the same for now, our team was recently "re-org'd" into the Cloud Computing Marketing group at Sun. As such, I quickly learned that to those outside of our industry, they usually have no idea what I'm talking about when I say I'm in the "Cloud Computing" group. "Cloud" is of course the buzz word at the moment, but it's still not a term that has permeated much outside the tech realm. My colleague, Neave Connolly, was facing the same challenges describing "Cloud" to her friends and family, so she undertook some research to look for a good, simple definition we could use for the non-techies in our lives (yes, this means you, Mom ;).

Neave found some great information so I thought I would summarize and share.

First, though, I often lead with the "utilities" analogy, as that usually helps. (Remember when Cloud Computing was pretty much "Utility Computing" a few years ago?) It goes like this: When electricity was first invented, companies would have to build their own power plants and hire their own electrical engineers to manage them. After time, electricity became a utility, and companies were no longer burdened with providing it themselves. Computing is (in theory, at least) following a similar pattern. Why should every company have to build their own data center and hire their own system administrators? Why not buy computing as a utility delivered over the Internet, just as electricity flows through wires into our factories, offices, and homes?

And that brings us to the Cloud Computing articles for those who'd like to understand a bit more...

• The first reference is a good, simple, high-level overview and definition of Cloud Computing from TechTarget.
• Next up, Gartner Highlights Five Attributes of Cloud Computing.
• And finally, there is a great deal of Government interest in CC for the efficiencies it can offer, and the National Institute of Standards and Technology (NIST) has weighed in with their Working Definition of Cloud Computing.

All the articles basically define and describe these key attributes of CC:

1. CC is sold on demand. In many cases, you might only need a credit card to buy computing power and services.
2. It is "scalable" and "elastic," meaning you can use as much or little capacity as you need, there is no upper limit on what you can use, and you pay only for what you use (just like electricity).
3. It is provided as a service, fully managed by the provider. The customer need not know "what's behind the curtain," just that they are getting a service that meets their needs. (Sticking with the analogy, do you know what power plant provides your electricity? And even if you did, do you know how many generators are in it, what their capacity is, etc.? Do you care?)
4. The "Cloud" is shared. Multiple users may be serviced from the same infrastructure and systems (virtualization being one key, but that's another tricky definition I don't want to get into right now!). 
5. It's Internet based. All services are managed and provided over the Internet using Internet standards and protocols (though as a still-evolving technology, there are bound to be new CC-specific standards emerging over time).

There are many interesting debates ongoing about what is or isn't CC, how does or doesn't it differ from "older" technologies like Grid and Utility Computing, where's the line between "public," "private," and "hybrid" clouds, etc. It just points to the youthfulness of this technology, while the huge amount of interest, attention, and debate (within our industry, at least) point to the almost unlimited potential this "paradigm shift" in computing holds. Will company data centers go the way of company owned power plants? We'll find out in time, but meanwhile, you hopefully have a better idea of what is meant by "Cloud Computing"!

I first wrote about our forays into "1-click" downloading a while ago (could it really be two years already?), and as that post noted, there were significant limitations in our first 1-click widget. We had to take on some higher priority projects that delayed our ability to address the limitations for a while, but as you'll see, we're now solidly back on track. 

While I don't work full-time on downloads any more, I am still the Product Manager of the Sun Download Manager (SDM) and of course remain very well-connected with the Download Team. As such, I was asked by Alfred Chen, the current Download Team Lead, if we could try out the latest version of the "advanced download widget" (ADW) on the SDM download page. SDM remains one of our most popular downloads so would get plenty of traffic for "real world" experience and also met the criteria for the new ADW. I was glad to say "Yes," and we went live with it several weeks ago. 

One of the main limitations of the original 1-click release was that it worked for one file and one file only. The new ADW now accommodates multiple platform and language options so is much more versatile.  It even "auto-detects" your platform and language and will pre-select the appropriate option when available. Once platform (and if available, language) are selected, it automatically pushes the correct binary for 1-click downloading. Here's a screen shot of the new widget:

You can try it out for yourself by downloading SDM.

There's more work to do, however -- ADW still does not support optional or required registration, so that's the next set of enhancements. They are underway and if all goes well will be released this summer. I have previously touted the virtues of optional registration, and there are several places on Sun now where we're using this with very good results. For example, on the Glassfish Download Page, there is an optional registration "pop-in" widget that comes up when you click the Download button. This illustrates a new twist on "optional download registration," as it includes the added incentive of receiving an informative white paper for registering. Being optional, you can just close this out and the download will start automatically (there's that 1-click functionality again!), but many customers are taking advantage of the white paper offer. So they get a white paper and the Glassfish download, and we get valuable customer information. 

While this new model is working great, the Glassfish implementation is frankly a "one-off". What the next version of ADW will provide is a standard way of doing this across Sun sites with minimal effort. I'm really looking forward to this new functionality and think it'll be a great win for both Sun and our customers.  It continues the march towards our goal of seamlessly integrating the download experience directly into the web pages you're on, rather than forcing customers to "move" between the web site and the "download system." 

One way to develop a better understanding of the customer's needs is to simply ask for their feedback.  Through out sun.com, you will notice a floating [+] icon in the lower right corner of your Web browser window.  When you click on this icon, an OpinionLab scorecard pops up allowing to you provide input on your Web experience as it relates to the content, design and usability of the current page.  Below is a screen snapshot of a sample scorecoard:

Recently, we added the OpinionLab icon to the Web pages for the download application that powers the Sun Download Center.  In the month of April alone, we received 481 ratings through the OpinionLab scorecard.  90 people chose to provide additional feedback through the Comments field. The overall rating was 3.5 out of 5.  It was a bit lower than I expected.  Again, this is why we're asking for your input, so you as the customer can help drive further improvements base on your needs.

I would like to thank you if took the time to provide us with your feedback on the Sun Download Center.  While I cannot share the specifics of the feedback that we received through the Comments field, I can say that most of the comments were fair.  I'm still flattered by the number of people who are fans of Sun, our products or the Sun Download Center.  Regarding the constructive feedback related to the download experience, if it's actionable, we will definitely include the enhancement in one of our upcoming releases.

In my last post, I displayed an image of a new display widget for personalized content on sun.com web pages. I noted it was a draft, and with good reason -- nobody was ready to sign off on the design at that point! The main issue was that it took up a considerable amount of the most valuable "above the fold" space on sun.com pages, and the content owners weren't enthralled with having their content pushed down the page. Our excellent design team mulled this over quite a bit and came back with what we feel is a great compromise design. (I am a big proponent of "compromising" and try to do so whenever reasonable, so that all parties feel relatively satisfied with an outcome. That's probably one of the reasons I'm about to celebrate my 24th wedding anniversary! But I digress...)

Here's a wire frame of the new solution. ("Wire frame" refers to more of an "outline" picture of the solution, not the final coded and realized design. Ignore the numbers -- they cross-reference content in a design specification. While we have the final design complete and coded, I'm not at liberty to preview it publicly just yet. Suffice it to say this wire frame has been translated into a very cool new widget!)

So here we see the recommendations as more of an overlay rather than a full page-wide component. This offers the immediate advantage of making it readily clear there is content behind the widget (and in fact you can still see the whole left hand top of the page), and no content had to be pushed down the page. It's also attention grabbing, and we do want customers to notice the personalized recommendations we have for them. We keep the fashionable "carousel" functionality so that you can click to see more recommendations, and we make it readily apparent how to close the recommendations to view the content underneath. Once closed, it'll remain in a closed state until the user opens it again or until the cookie that tracks this preference expires or is removed.

I'd be remiss in not acknowledging and thanking the design team that worked so hard to find not just a workable solution, but what I think is an excellent solution as well: Margaret Brown, Chris Haaga, Sara Shuman, and Andrew Payne.

Now that I've whet your appetites, you're probably wondering when this will be live. Sorry, I can't publish dates for functionality that we're still building, but suffice it to say we're making great progress.

I wrote an intro previously on SMILE, the Sun Machine Learning Engine, our new personalization system that utilizes predictive analytics. It went live in January, 2009, and we've been carefully measuring results since. So far, we are seeing higher click-through rates for SMILE served ads versus default ads, which is what we hoped would happen. But overall click through rates on these ads, whether personally targeted via SMILE or simply default, are very low, so we need more bang for the buck. We believe that the small ads served by SMILE in the right-hand navigation are easily overlooked -- many users simply ignore banner ads and/or the right hand column, concentrating on the central content of interest.

So our next step is to provide personalized recommendations in the body of the page. Here is a draft mock-up of a "carousel" type approach we hope to use:

There are numerous challenges with this approach, as the page real-estate "above the fold" is highly valued, but that's where we need the recommendations to be if we really want them to be noticed and of value. A likely approach will be to make the carousel collapsible, so users can close it up if it's in the way, and then open it to look for personally recommended content. We'll also provide a new "All Recommendations" page that is solely dedicated to showing all recommendations for visitors.

We did a usability study recently to test these concepts and were pleased with the results. Customers told us:

• They like what we're planning. We have literally millions of web pages, so if we can use our analytics to help customers quickly find the products, services, and content of most interest and value to them, we are saving them time and making them more efficient. Definitely a "win win" for all of us.
• They would likely not notice the recommendations if they're not centrally located high on the page.
• They don't have serious privacy concerns. Because we are an enterprise business site, they expect us to use analytics to improve the experience, and they'd appreciate the benefit. They would look at things differently if we were their financial institution, for example.
• If the recommendations are not accurate or useful, they will quickly learn to ignore them. So the onus is squarely on us to ensure our system is up to the task of making accurate, valuable recommendations that meet our customers' needs.

This is an ambitious undertaking involving a whole bunch of teams across Sun, from Design to Publishing to Engineering to Analytics, so it's at least a few months away.  I'll keep you informed of progress and of course announce when you can check out your own personal recommendations!

I've actually been on my new job for a number of months now, so it's time I start writing about it! After 10+ years working in the ESD (electronic software distribution) space here at Sun, it was time for a change, and so I took on a new challenge in the eMarketing realm. I am overall Program Manager on the sun.com web team for an initiative we call "Contacts to Revenue" (C2R). The essence of C2R is that we have zillions of web visitors, and especially downloaders, and we need to do a better job of identifying contacts and hopefully "nurturing" contacts into customers. There are several primary initiatives now underway.

First, we're creating new content we feel is of value to our customers and asking them to login to receive it. (If you don't already have a Sun Online Account, then there is a brief registration form that must be completed.) We feel this is a fair value exchange -- you tell us who you are, and we provide valuable information in return (examples include white papers, blue prints, webinars, and downloads). Here's an example:  Deploying Hybrid Storage Pools with Sun Flash Technology and the Solaris ZFS File System.

Second, I want to discuss Project SMILE -- The Sun Machine Learning Engine. We're building a new analytics engine to help provide more relevant content to web visitors. I'm the Project Manager for SMILE and pleased to say we went live with "phase 1" in mid-January. For the initial release, our goals were modest, focused primarily on improving the relevancy of small advertisements displayed on our main web site. We used a good ad banner server program previously, but it lacked sophisticated segmentation and targeting capabilities, so we built our own. If you are a return visitor to sun.com, we may serve you small ads that are more targeted to areas in which you've indicated an interest. All the ads are unobtrusive, appear in the right hand column, and are of this format:

For SMILE Phase 2, we're working on the ability to more proactively serve "Recommended Links" and other content of value based on web visitor's interests in our products and services.

We realize that asking for a login to obtain content and using machine learning and analytics to serve customers may be of concern to some. However, the bottom line is that we're in business to make a profit for our shareholders. Times are tough (duh), and it's imperative we do a better job of converting our web visitors to customers in order to grow our customer base and increase revenue. It's not a one way street -- we will continue to provide exceptionally valuable and unique content in return as well as striving to make relevant content more apparent and easier to locate on the web. 

In conjunction with these changes, we have recently updated our Privacy Policy to more accurately reflect how we'll be using customer data in support of these programs (among other updates). Please read it if you wish to be fully informed on this subject. (We also make it possible to opt-out of web personalization if you wish.) In any case, as this blog post and the new privacy policy illustrate, we're openly communicating about these new programs and changes. Our intent is to better serve our customers and our shareholders.

(This post covers the last day of the three day conference. Also see: Day 1, Day 2.)

The final day kicked off with a "web meets food" theme as Michael Pollan (noted author focusing on "the places where the human and natural worlds intersect") took the stage. His was not a cheery message for first thing in the morning. He noted that our current methods of food production are large contributors to green house gases, with a heavy reliance on fossil fuels. He feels we can and must turn agriculture into part of the solution, not the problem, and that we'll all be much better off and healthier if we "eat sunshine, not oil." A big part of the problem is we're locked into a cycle whereby the more we process food, the more profitable it is, but the less nutritious. 

He suggested ways the web can improve the situation. First, make the food system more transparent. If we really knew how food was made and its impact on the environment, it would be a major catalyst for change. Secondly, the Web is the perfect place to form communities of like minded people who can focus of enabling sustainable alternatives as well as cultivating and identifying local food sources. Just the cost and impact of transporting food is huge, that's why farmer's markets may well be a model of the future (back to the past?). Is it smarter to bake butter cookies in Denmark and ship them to New York, or to ship the recipe to New York and bake them locally?

Back to more mundane matters (like making money), Richard Rosenblatt of Demand Media Inc. gave a very impressive talk and demo of his company and its new Pluck on Demand product. They make it incredibly easy to add contextual content and social media to any web site, in a self-service model. (I saw a guy in the row in front of me on his laptop doing this real-time to his web site once he saw the demo!) Richard explained some of the driving concepts behind his vision and services:

• In a search driven world, audiences fragment but then reform around passionate verticals. Provide the tools to engage them quickly in your communities once they find you.
• Social media will be the key success factor for every web site.
• Look to grow organically via community, not just search.
• Create content at scale, leverage the community content model. 
• Support it with algorithmic, ROI driven content placement.
• The right content + social media = user engagement and revenue.

Reinforcing that "cloud computing is all the rage," we had our second panel on the subject, featuring Padmasree Warrior (CTO, Cisco) and Shane Robison (CTO, HP) in a conversation with Tim O'Reilly. They started by discussing the changing role of the CTO, which is more focused now on strategy and the future of business technology. (That was contrasted with the CIO role which they described as more operational, often implementing what the CTO recommends.) And then onto the benefits of the cloud...

• It's the next evolution of computing.
• Applications are abstracted from physical location.
• We haven't hit the tipping point yet. We still need "hybrid clouds" that incorporate standards and can interoperate with "private clouds." Think "inter-clouds" as opposed to inter-net.
• HP wants to provide "infrastructure as a service."
• Clouds can help with Green IT. Think of instrumenting the environment through the cloud.

For a conference that started the day after the presidential election, we finally got down to some politics. Enter author and mega-blogger Arianna Huffington, San Francisco Mayor Gavin Newsom, and high-profile political consultant Joe Trippi. It was a lively and spirited discussion, with everyone in a pretty good mood as you might imagine. A few take-aways:

• 1960/JFK -- the first "television" presidency. 2008/BHO -- the first "Internet" presidency.
• Huffington stated something Al Gore would also say later in the day: Obama would not have won without the Internet (for fund raising, communication, and community building).
• There was a concept that the 'net "surfaces the truth," and perhaps Right and Left are outdated terms. What's important is where the truth lies (and it's not necessarily in the middle). 
• This could ultimately affect the future of our two party system.
• You can "fake it" on a 30 second TV ad. It's harder to do 24x7 on the Internet.

I thought Elon Musk provided the ultimate example of living your dreams. He said that during his college years he felt there were three important problem areas for focus: the Internet (he was a founder of PayPal), clean energy (see Tesla and SolarCity), and space (SpaceX). He had an unassuming manner about him, but you can see he's really putting his time and money to work on these problems. Compare that to making your fortune, then kicking back on your own tropical island...

"The Platform" had some very interesting dynamics when you consider the fierce competitors sharing the stage: Moderator Max Levchin (Slide), Vic Gundotra (Google), David Treadwell (Microsoft Live), Amit Kapur (MySpace), and Elliot Schrage (Facebook).  Fortunately, peacefulness reigned, though there were a few testy moments (mostly Microsoft v. Google, as you can imagine). The main theme was how to evolve the platforms and support the developers:

• Enable developers to build viable biz models while increasing customer engagement. ("People like to share information.")
• Join people, places, devices, & applications.
• The Platform must benefit all three to survive: provider, developer, and customer.
• How do you deal with fast evolution vs backwards compatibility (especially for your 3rd party partners)?
• Build strategic alliances
• Be transparent and open to feedback
• Provide tools, resources, and support
• Rely on the evolution of the Internet to push forward the platforms built on it.

Bob Sutor, IBM VP of Open Source, said the world is at an "inflection point" where we must start using IT power to start solving world problems. Make big bets, public/private partnerships, and open standards are key.

I was fascinated with Shai Agassi's talk about building an "electric car network" (see BetterPlace). This is truly a big bet! Imagine an electric car that you pay for by the mile, and you don't own the battery. Need a charge? Pull into a "filling" station -- they remove the dead battery, put in a charged one, and off you go in less time than it takes to fill your tank. How fast this can happen is a "matter of policy" (and some deep pockets I presume), though he said they are actively working in Israel to release the first network.

"Track Me" was a panel on geo-location and some of the cool things happening now and possibly in the future. The panel was composed of moderator Brady Forrest (O'Reilly Media), Greg Skibiski (Sense Networks), Ted Morgan (Skyhook Wireless), April Allderdice (MicroEnergy Credits), and Rich Miner (Google).

There was a fascinating demo of the kinds of things Sense Networks can calculate and present, based on "analyzing massive amounts of real-time location data to understand and predict aggregate consumer trends." You'd be surprised by the correlation between the time and location of taxi cab trips in San Francisco and the rise and fall of the stock market! They have a lot of data and are still trying to figure out the meaning of some startling results and trends, plus ways to "segment" based on location.

Miner from Google talked about building location into the platform, then letting developers build upon it. Privacy was an obvious concern that came up. People need to own their data and have the ability to control, review and delete their location data. What is the geo-location equivalent of a cookie?

Rebecca MacKinnon (Global Voices)  and Isaac Mao (currently a research fellow at Harvard Law) talked about overcoming censorship on the Internet (mostly in China), and its impact on closed governments. McKinnon talked about the difficult place Internet companies find themselves, wedged between governments and individuals. Mao introduced his concept of "sharism".

And with that, it was time for Al Gore, the exciting grand finale. I really enjoyed seeing him in person, and he held his own quite well, showing an impressive capability to talk tech and appeal to the audience. As mentioned, he reiterated the message that Obama would not have won without the Internet. He talked about Current TV and their efforts to "democratize the TV medium." He used some powerful analogies:

• The invention of the printing press almost 600 years ago "freed" information for the first time from the elite and created a new information ecosystem. Television "refeudalized" the control of information into a few elite hands. Now the Internet will democratize information again.
• We are moving to a time when Web 2.0 will be taken for granted, like a "fish in water." He challenged us to move to "World 2.0."
• Finally, a great analogy that really resonated with me (showing my age, I guess):
  JFK took office in 1960 and said we would put a man on the moon in 10 years. It seemed incredibly ambitious and perhaps unattainable, but it was done in a little over 8 years. He challenged Obama and all of us to take on a similarly ambitious goal: Make 100% of all electricity in the US from renewable energy within 10 years. If we could go to the moon back in the 60's, we can do this in the 2000s!

The conference came to an end with a final reception (i.e. yet more drinks and hors d'oeuvres), and everyone I spoke to thought it was an excellent event. I agree. If you've read this far, I thank you and hope you found this overview provided a decent taste of an unusual and exceptional conference.

(This is a continuation of my prior post, "Overview of  the Web 2.0 Summit 2008.")

Day Two started with Paul Otellini, President and CEO of Intel. He focused on two big areas of opportunity that Intel sees. The first was "enterprise" (or professional) social networking. He showed a thought-provoking demo of the unified enterprise community network of the future -- tightly integrated communication tools, professional learning management, robust search and RSS feeds. The demo focused on a fictional Intel new hire in Asia and how such an enterprise web community would smooth her integration into the company and also accelerate her learning and effectiveness. Then he said something like, "Of course, these tools don't exist yet..." 

The second big opportunity was the "personal Internet," where the machine brings relevant and timely info to you rather than you having to seek it. This led to another demo, this time of the smart mobile device of the future. Features included machine translation (written and verbal) in the palm of your hand, instant commerce, lots of geo-awareness, etc. It demonstrates Intel's focus on bringing the power of the web to mobile devices.

Next up was Kevin Kelly, a long time reporter on the culture of technology who helped launch Wired Magazine. This is more or less what his first slide looked like:

Care to guess the meaning?

Turns out it had been exactly 6,527 days since Tim Berners-Lee transmitted the first web page. The Internet (r)evolution since then has been amazing, but Kelly really said we need to hasten the next phase, the so-called Semantic Web. It's about awareness and linking of data. First we linked computers, then pages, and next we need to link data. 

So what will the next 6,500 days bring? Kelly feels it'll go way beyond "the web, only better." He spoke of one large machine of interconnectedness, the Web Operating System (I couldn't help thinking of Sun's seminal mantra, The Network is the Computer). The web will "own" all data -- if it's not part of it, it doesn't exist. All devices are connected, processed, and structured. One media platform for all. Everything is always on. "Believe in the impossible."

This could be an awesome thing, though it does have that "big brother" ring to it as well. Let's hope the "security guys" get it right.

The plethora of iPhone enthusiasts in the audience were entertained by a talk with Ralph De La Vega, CEO of AT&T Mobility and Consumer Markets. Let's just say he's pretty happy with how things are going. 

The web can and will play a huge role in the future of health care, as discussed in the "Future of Health" panel. Lots of talk of genetic testing and online communities. Check out 23andMe, then order your own genetic profile for a mere 400 bucks.

Appearing in full battle fatigue, Army CIO Jeffrey Sorenson showed the military side of social media. The Army lags behind the speed of commercial innovation and is just rolling out Web 2.0 now. Security concerns are paramount of course. But we saw interesting demos of "Battle Command Knowledge System" and "Command Post of the Future" -- web based systems that will greatly enhance our military capabilities and speed/accuracy of communication.

The "Future of Media" panel provided an engaging discussion featuring Evan Williams (Twitter), Joel Hyatt (Current TV), and Ken Auletta (The New Yorker). I remember coming out of my first Web 2.0 Expo in 2005 thinking I had to start a blog (which I did). This time, it was all about Twitter, so I'm going to give it another try too! (I'm admittedly not 100% convinced, but time will tell.) They talked about Twitter changing how people connect --> will it actually change our culture over time? 

Today we are enabling "conversations" in every possible format, though the questions remain on how to monetize these new interactions. (They didn't have the answer, sorry.) 

I found Hyatt's thoughts on Current TV provided a refreshing perspective:

• Many people innovate, then look for a business model. They did the opposite.
• They're developing "user generated advertising" (he showed a clip of a Toyota commercial made with a hand held video camera). Their study shows these ads are "preferred 9 to 1."
• The networks are focused on moving TV to the Internet. Current is focused on moving the Internet experience to TV. 

Next speaker, Saul Griffith, a rocket scientist type for sure, reminded us that we all contribute to the energy crisis. We all have to help solve it. He's built a very clever web site that lets you easily determine how much energy you use in a day. He suspects we'll all be surprised and ultimately better conservationists after finding that out.

I have a 17 years old son, so couldn't help noticing the physical similarities to Mark Zuckerberg, founder of Facebook. Yes, he's young, but he was very composed and well-spoken. (BTW, it's been a big week for 24 year olds around here!) He took some heat for the closed nature of the Facebook platform, but that's changing, and he said there is a natural progression from closed to open systems over time. (That's a key part of Sun's open source vision, so let's hope he's right.) He spoke of a long term goal to enable people to share information any way they want on any device. How to make money? Think of how to best entice customers to share information, then deliver "engagement ads" that are highly effective. 

A very high powered (pun intended) panel was up next on Cloud Computing -- Paul Maritz, CEO of VMware, Marc Benioff, CEO of salesforce.com, Kevin Lynch, CTO of Adobe, and David Gerard, Google. You know this is one of the hottest topics of '08 and extremely well covered, so I don't think there was a lot of news coming out of this. Some interesting concepts were mentioned though:

• Information will outlive devices and applications; the cloud becomes the "information bank."
• Providers have to protect, enable access, and add value to information stored with them.
• We're hitting the "fourth generation" of software, getting back the richness of client-server applications (which we lost with Web v1.0) but keeping the power of the Internet.
• They need to provide the tools to enable the movement of apps to the cloud.
• We need to break down the disconnect between rich customer experiences and pedestrian enterprise experiences.
• What if you had a rich Flash interface to SAP? (Adobe wants to be the "interface to the cloud.")
• It doesn't have to be a low margin business. Differentiate by providing unique value and/or a large ecosystem.
• Clouds are still proprietary, though. More value will be created as they interoperate and provide new services. (We're seeing this now with Salesforce's integration with Google Apps, for ex.) 
• Standardization is needed, as well as new data models that break down data silos.
• Eventually, consider we'll have "personal clouds" as opposed to multiple friend lists on separate web sites.

Shifting back to the future of advertising, there was a talk with Jack Klues of Vivaki -- they control billions and billions of dollars in ad budgets for some of the world's biggest brands. His message for weathering the economic storm:

• Maintaining and protecting brand is at least as important during a downturn as any other time.
• You can increase share as media buys become cheaper.
• Don't lose sight of what people care about; keep on message and brand.
• Continue to look at how to optimally blur the lines between advertising and content (ethically, of course...)

A "Launch Pad" event was next. Five start-ups gave their elevator pitch to a panel of VCs. The audience voted on our favorite by texting votes which were tabulated and projected real time. I agreed with the winning choice, GoodGuide. Great concept and valuable info -- take a look. (No real losers here, all were of interest but not sure they'll all survive: Predictify, Sungevity, Everyscape, Qik.)

The day closed with a Music discussion with Chris DeWolfe, co-founder of MySpace, and Edgar Bronfman, CEO of Warner Music. It was a bit too much focused on the launch of MySpace Music I thought. They said the music industry will be "very different in 5 years," but not necessarily how. And with that, another long but stimulating day ended.

Continue to read about Day 3...

I felt fortunate to attend my third Web 2.0 Summit, November 5-7, 2008, in San Francisco (thank you, Andreas!). I really thought it was an excellent event and fairly different from the prior ones I attended (here are my comments on the 2005 Conference and 2006 as well.) What was most different was the strong and well-coordinated thematic focus on "World Meets Web" -- it truly permeated and unified the sessions and discussions. The concept behind this is that we've come a long way with the web and it has become a major influencer on the global society. How do we use this new found medium for the betterment of the planet and to deal with all the issues we face? It couldn't have been a more timely theme,  nor punctuated better than Al Gore's closing comments, as he noted Barack Obama could not have been elected without the web! Talk about web meets world....

The format was generally the same as previously. Wednesday morning provided the only workshops, where you have the choice of attending several tracks that run simultaneously. (This is where you typically wish you could be in two or more places at once so you don't miss anything.) Starting Wednesday afternoon and through the conference closing late Friday, we all assembled in a large conference room for the engaging interviews with true industry leaders as well as the “high order bits” (short, 10-20 minute talks by various domain experts).

There's lots of info online of course about the Summit, and in the interest of time and efficiency, I'm not going to bother deep linking to everything. Here are the bios for all the speakers, and many of the presentations (and video) are online too. My main goal is to share some of the insights I captured from the various speakers and events.

For the workshops, I chose topics related to presentation and advertising, as this relates closest to what I'm working on now (developing new web personalization and ad serving capabilities for Sun).

The first talk was a bit of a strange start, as a little ways into it I realized it had nothing to do with the web (at least directly). Go figure. Nancy Duarte's topic was "Telling Meaningful Visual Stories" and was really about creating compelling presentations. It was targeted partly at least to the many entrepreneurs that attend, with lots of guidance on how to make compelling pitches to VCs. But I think there are lessons to be learned for presenting info on the web as well. Some key takeaways:

• To connect with the customer, become a story teller. Share experiences, be vulnerable perhaps, believe that your own story can touch customers. (This is something that cannot be outsourced!)
• Consider your audience first and how to reach them, then develop the content. A presentation is not a “document”.
• Break out of the mold – start with a blank page, not a template. Change your environment, get a different perspective, collaborate and brainstorm on ideas. You want to inform, inspire, persuade.
• Avoid complicated architectural diagrams, as they don't resonate. "Tell" your architecture as a story.
  
• Use metaphors. Include conflict to grab attention. Produce ideas and stories, not slides. Be transparent, be different. Your stories can change and influence!

The next workshop I attended was "Consumer Brands Tackle Marketing 2.0," with some high profile marketers from leading companies. Some nuggets from that panel:

• Technology changes, people don't.
• Social media offers new opportunities for gaining customer insight, if you know what to do with it. Consider "perpetual marketing," as the engagement doesn't end. We're still evolving how to use social media consistently in marketing programs.
• The audience defines the message. Good creative still wins, technology can't overcome crappy messaging and content.
• Traditional media has the reach but many in the audience won't be interested. For example, if you advertise Pampers on national TV, what percentage of the audience cares? This is really where we can differentiate advertising on the web, by targeting interested and relevant audiences much more closely.

As I mentioned, we're working on more personalization initiatives here at Sun, and one outcome is that we're bumping up against our own privacy policy. There's a fine line there about what we can and cannot do. I asked the panel about how they deal with it, and mostly got shrugs! David Knox from Procter & Gamble did say they had just changed their privacy policy to enable global opt-in/opt-out on their site, instead of by section as it was previously. Their approach was to email every single customer and inform them of the new policy.

So it seems everyone is trying to deal with this, and there is no perfect answer. I think the best thing is to be transparent with what we're doing. We need to revisit Sun's privacy policy to see how it can evolve to match the evolution of our web and personalization capabilities. When we do make a change, we'll need to communicate it appropriately.

The last workshop I attended had the not-so-helpful title of "Please - Not Another Network Panel." Sponsored by Yahoo, it actually focused on web advertising and ad networks. I'm pretty new to this area so didn't follow all the subtleties of the conversation about how these mega-networks work, but I managed to learn more about that side of the industry.

And with that, it was lunch time, then we headed in for the start of the mega-conference. The focus on web meets world was clear from the start, as the first interview was with Larry Brilliant, head of google.org, Google's philanthropic arm. Google committed from the start to putting 1% of revenue, profit, and resources into philanthropy. We'll see this theme repeated throughout the conference – the web has made you millions (or billions), so how will you use that to make the world a better place?

Google is focused on three key initiatives:

• Inform & Empower -- mostly a third world focus on empowering communities to get better information and accountability from their governments.
• Predict & Prevent – Early disease identification and prevention. This touched on another recurring theme, “instrumenting the world.” We have great new sensor capabilities and can tie the sensors together via the web for almost real-time, unprecedented insights into the global ecosystem.
• Renewable Energy -- specifically, we need a form of renewable energy that is cheaper than coal. At that point, there is no financial barrier to adoption.

Bottom line: "It's time to use technology to make the world a better place."

Mary Meeker gave her usual dizzying blitz of a zillion financial slides in about 15 minutes. It's pretty hard to digest that quickly but my takeaway was that, while the economy may be tanking, there is still plenty of growth and opportunity for Web innovators.

Rajesh Jain from Netcore Solutions (see his blog at emergic.org) explained how there are about 10 times as many people with cell phones in India as have Internet access. As a result, SMS is huge, offering ease of use, low cost, and easy value added services. There's a huge direct to consumer reach. He used the term “invertising” for invited ads in SMS messaging.

Up next was super-VC John Doerr, who helped fund Sun (amongst many other companies) years ago. He talked about Obama being the first President who will have a CTO, and suggested (Sun co-founder) Bill Joy would be a perfect fit. He felt energy research is our #1 priority and expressed a big concern that we're not graduating enough scientists and engineers. He also felt it was absurd that we train foreign students, then kick them out of the country once they're trained. I loved the way he summed it up:  "We should staple a green card to their diplomas!" Makes sense to me.

Doerr offered up survival tips for start-ups during the economic meltdown (which he described as more than bad mortgages, it's a "crisis of confidence"). I think much of this is great advice regardless of how big your company is:

• Hunker down, take on the long view, focus on your core
• Act now, and with haste (before funding gets cut)
• Protect the core of your business (don't lay off the key people!)
• Have 18 months worth of cash in the bank
• Defer facilities investments
• Re-evaluate R&D priorities
• Renegotiate all your contracts
• Everyone in the company needs to SELL
• Give equity instead of cash
• Secure the cash (i.e treasuries, somewhere safe)
• React quickly to leading indicators and KPIs
• Over communicate, don't sugar coat bad news

Next up, Tony Hsieh, the personable CEO of Zappos.com. I think the Zappos story is well-publicized so won't go into details, other than to say it's quite impressive what they've done. This web-based "shoe store" is now a $1B business and expanding quickly beyond footwear. A few key points:

• The company is "powered by service".
• The goal is to WOW each & every customer.
• Business is built by repeat customers and word of mouth.
• It's good to have people call your customer support line. There is no better branding opportunity then speaking with your customers.
• Company culture = company brand.

Larry Lessig, the outspoken Stanford Law professor, talked about the current failures of our government. He says it's come to the point where the Number One job of our elected representatives is to get "tenure" (i.e. raise enough money to get elected over and over again). Money destroys trust. We must change congress to fix our economy.

Day one ended with Yahoo CEO Jerry Yang on the hot seat, grilled somewhat mercilessly (but fairly, I think) by conference promoter John Battelle. Jerry took the hard questions graciously and stood his ground. He explained Yahoo's strength as a consumer brand that enables people to get what they want from the 'net. It's the starting point.

He talked about recent efforts to open up the Yahoo platform with the Yahoo! Open Strategy, enabling third party apps to integrate with Yahoo services. There is opportunity for all in the creativity that will come out of opening up the platform.

What I hadn't realized was the large press presence and how they would pick up on what was being said. While discussing Microsoft, Yang said he thought Microsoft should buy Yahoo. I was surprised when this casual statement during a conversation turned into national news.

As you can see, it was quite a stimulating Day 1!

As an aside...
I was amazed at the proliferation of iPhones in the audience (and Blackberries -- the lady sitting next to me had one of each on her lap) and somewhat dismayed at how many people were looking down constantly with an eerie blueish light on their faces from the phone displays. It didn't seem to me they were paying attention to the conference, and I wonder about this newest contributor to attention deficit disorder. But perhaps they're just better at multitasking then I am.

Continue to read about Day 2...

Here's a "best practice" that is a source of never-ending debate: Should we require login/registration in order to download our products? There is no right answer, but there are plenty of pros and cons.

Sometimes, we have no choice but to require login in order to comply with government export regulations. (For example, if a product contains some types of strong encryption, we must keep a record of who has obtained it.) An obvious reason in favor is that knowing who downloads our software is of great value in follow up marketing initiatives as well as demographic analysis of our customer base. It's not all a one-sided equation though. For our customers who login first, we offer a useful download history function, and we also automatically provision our My Sun Connection portal with a list of products of interest to you. (This automatic portal provisioning is set up for many, though not all, of our downloads.) It's very handy, as you can click on product names and quickly see a valuable list of related resources and information. (I encourage you to check out the portal if you're not familiar with it. Once there, be sure to click the "My Products" tab to see how we aggregate product information from many resources in one place!)

The other side of the coin is that requiring login or registration before downloading definitely hurts the conversion rate. Using our web analytics, you can see users drop off with every extra click, and the largest abandonment is at the login/registration step. In many cases, the value of getting our software into the hands of our (often first-time) customers as quickly and with as few roadblocks as possible trumps the value of collecting user information, so we do not require registration. 

With such conflicting agendas, you must be wondering by now what I'm going to recommend as the best practice? Well, that's easy, don't use either -- use optional registration! This feature provides a middle ground that works for most products -- just give customers the choice. If they want to login to gain the benefits offered, we welcome and appreciate it and believe it's worthwhile. And if the customer just wants the software as quickly as possible, there's no need to register, just click Continue. Here's what you'll see when you request an "opt reg" product on Sun Download Center:

For those who do choose to login first, we find a better quality of data since they are doing so of their own volition, and it allows us to build a stronger relationship with the customer. Of course, we have to face the facts -- a majority of customers will skip the login step and click Continue. For that reason, we highly recommend building optional product registration into the product's installer as well. Once you have committed to installing the product, there's an even stronger opportunity to build the relationship and help support the software on your system.

Fortunately we built our download system to support all three options (required, optional, none) easily, so we can meet whatever the product team wants with the flip of a bit in the product database. However, our recommendation remains to use optional registration whenever possible.

This is the third in a series of download best practice posts.

• Download Best Practices, Part 1
• Download Best Practices, Part 2

It has been nearly 2 months since I last blogged.  I was sidetracked by recent changes at work and in my personal life.  In my expanded charter at Sun, I'm now responsible for defining and implementing a  2-3 year Download platform strategy and road map for supporting Sun's business model on the Web.  At home, my 10 month old baby girl started attending daycare 3 weeks ago.  As predicted by my friends with children in daycare, my baby girl got sick within the first week.

When work and personal life gets hectic, it's easy to loose focus on our surroundings including our families, friends and the broader community.  With the recent meltdown in the stock market, undoubtedly everyone's mind is on their investments and retirement funds.  The going joke is that the 401K should be aptly renamed the 101K.  While the government appears to be aggressively tackling the financial problems, the looming news of a major recession is having an even greater impact on the less fortunate people.

This past week, a small group of Sun employees volunteered at the Glide Memorial in San Francisco.  We spent 2 hours serving lunch to the poor, the elderly, and the homeless who depend on Glide for 3 nutritious meals a day.  The door at Glide is open 365 days a year, and it's open to everyone.  During lunch alone, they serve up on the average about 700 to 900 meals.  But without funding from local government, grants and individual donations, as well as help from volunteers to help serve the meals, this non-profit organization cannot perform its miracles.

I signed up for the volunteer event, because I needed a break and  because it was an opportunity to bond with my co-workers.  I walk away with a new perspective, with greater appreciation for my own welfare and greater recognition of the needs of the less fortunate.  I wanted to spread the word and encourage you to consider how you can help during such difficult times.  Lastly, if you have never volunteered at Glide, I highly recommend that you try it at least once.  I walked away feeling really good about my service.

After more than 10 years of focus on download systems here at Sun, I was ready for a change of pace (as you might imagine). Once we got our new download system out the door, my manager was much more willing to consider a change for me, and then a great opportunity arose. I'm now managing a new project that is developing a machine learning engine for better targeting our offers to our customers. While the obvious intent is to benefit Sun by increasing the uptake on our products and services, we feel it benefits customers as well by providing more relevance to the ads and content we provide you on our web sites.

It's pretty much a whole new world for me, but I'm enjoying the challenge and learning about a whole new area of analytics. It would be a shame to ignore the experience I've acquired in the ESD realm, so I'm still participating in download strategy discussions and helping out where I can. I may well blog further on the subject as time allows.

But in any case, I am happy to see two new blogs about downloads and ESD that have sprung up recently. As I wrote in my very first blog posting three years ago, "I haven't found many resources for us 'ESD Professionals.' I'm not aware of trade journals nor web sites dedicated to ESD. I always enjoy meeting and speaking with my peers at Sun and other companies but wish there was more interaction, more sharing of best practices, and the myriad other benefits that come from knowledge sharing in a particular field. This blog is intended to be a small outreach..."

Well, I can't say I've seen an explosion in content, but I heartily welcome two new contributors to the ESD blogosphere!

• The overall Project Manager of the new download system was Alfred Chen, and he's the guy in charge now that I've stepped into a new role. Alfred is leading our Download Business and Strategy Teams, and we are very lucky to have him in those roles. Working with Alfred very closely for many years, I can only say he's a tireless contributor with exceptional knowledge, skill, and dedication. (And he used to be the manager of our download engineering team, so he's really one of those great persons with deep IT and Business experience.) So, to get to the point, Alfred's started blogging lately (about a variety of subjects) -- check it out.
  
• OMS Safe Harbor is one of the few companies out there with a major focus on electronic software delivery, and they've built up an impressive set of capabilities, products, and services. They recently let me know about their new ESD blog, and I've already read a number of interesting articles there. Great to see this outreach from OMS.

Mozilla and Firefox have garnered a lot of well-deserved publicity for their new Guinness record for most downloads in a day. This raises the question (again) of what constitutes a "download" and how is it measured? (I first touched on this subject a couple of years ago, writing "When is a download a download?".) I was glad to see in the above referenced BBC article, "The official figure was confirmed after logs from download servers were audited and checked to ensure duplicate and unfinished downloads were not counted." So that implies "attempted" downloads weren't counted, only completed. That's good, as I'm sure some stats we see reported don't make such distinctions. 

So with Guiness in the mix now, and with all the focus on downloads in the free & open source software world, I propose it's time for an agreed upon standard on how the number of downloads is reported. (To be honest, I'm uncertain if this could ever be a "formal" standard and who would govern it -- please let me know if you have any suggestions.) For starters, here's what I propose:

• Simple case: One Product = one file
• This case means that the entire product is contained in a single downloadable file. If it is downloaded successfully, the customer can install and use the product.
• The suggested measurement is the one generally used -- the last byte of the file is delivered to the customer.
• Note this may not be 100% accurate. Threaded download managers can conceivably download the last byte of a file and then fail an earlier byte range before the entire download completes. But this is an edge case. It would also be extremely resource intensive to have to match up all the byte range requests in a threaded download scenario to "prove" that one user completed the download. Thus, I think the "de facto" standard of last byte delivered is acceptable.

• Complex case: One Product = multiple files
• First, make the distinction between required and optional files:
• Required files are required to install the product. There may be one or more required files as part of the download. 
• For example, on Sun Download Center, there is an option to download Solaris OS as 5 CDs. If you do not download all five, you cannot install the product. Thus, all five are required files, and each one must be downloaded to install Solaris.
• Optional files are not required to install or run the product. Optional files may consist of "plug-ins,"  "add-ons," or "modules" that add functionality to the base program. Also included would be documentation, checksum files, additional language packages, and the like.
• The measurement of a completed download in this case is that all required files are downloaded by the same customer. (This can be a challenge to measure, but it can be done in most cases with a good degree of accuracy.)
• Optional files do not count and should never be reported as "product" downloads.

• In either case, "Attempted Downloads" should never be reported as a product download. 
• For a single file product, an attempted download is when the user starts the download but does not receive the last byte of the file.
• For complex products, an attempt is when any individual required file is not completed and/or the customer does not successfully download all required files.

First of all, congratulations to the FireFox Community on establishing the Guinness World Record for the most software downloads in one day.  As one of the 8,002,530 people who successfully downloaded FireFox 3.0 on June 17th, I feel privileged to be displaying the FireFox 3 Download Day certificate on my Blog.  After all, I simply downloaded the latest version of the Web browser (for my Macbook Pro) that I already use daily for all my Web activities.

Well, it has been 2 1/2 weeks and I have not encountered a single crash.  Performance has been great and all my existing add-ons upgraded without a hitch.  Thanks to FireFox 3 I'm enjoying my Web surfing experience even more so.  Kudos to the developers who work on the Mozilla projects.

There are so many new features in FireFox 3 that I simply haven't had the time to test drive them all.  Among the new features, my favorite include the Password Manager, Instant Web Site ID, and the New Location Bar with One Click Bookmarking.  I'm also a big fan of the FireFox 3 Download Map (Cool page, not a feature.) which provide an at-a-glance view of the latest FireFox 3 download stats across the globe in a very visually engaging manner.  I hope we can provide similar reporting UI for the software downloads at Sun.

Speaking of Sun, I was disappointed to not find FireFox 3 for Solaris readily accessible on the FireFox home page.  As it has been with prior releases, the download links to the Solaris port of FireFox 3 is buried in the Release Notes under Contributed Builds.  You really need to dig for it.  Well if you have been looking to download FireFox 3 for Solaris 10 or OpenSolaris, today is your lucky day.  I have conveniently provided the download links here on my Blog.

Download FireFox 3 for Solaris here:

Checkout the new Download Home Page on Sun.COM.  As one of the most frequently visited page, the newly enhanced design will allow customers to gain quick access to the most essential software from Sun.  These products are often the subject of Jonathan Schwartz's Blog.  Personally, I have recently downloaded OpenSolaris and xVM VirtualBox (desktop virtualization), and I'm now able to run OpenSolaris (and other operating systems as well) on my MacBook Pro using xVM VirutalBox.  How cool is that.  Anyway, I will be sure to blog about my virtualization experience in the near future.

Back to the new Download Home Page, if you do not find the software you need on the Featured Downloads tab, the new home page design offers numerous means to locate your software download.  You will find links to the New Downloads and Top Downloads.  Of course, navigation by Product Category or A-Z Listing is available as well.

Give it a try and download one of the feature products today!

I'm going to list three more of the download best practices that we encourage all our product teams to use as they develop products for electronic software distribution (ESD). This is the second part of a continuing series I'll publish from time to time. (You can see Part 1 here.) Some of these suggestions may seem like "no-brainers" (at least once you understand the basics of ESD), but they need to be stated and documented nonetheless and are certainly helpful to teams that are just starting out to prepare downloadable software.

1) Compress the files as much as possible. Obviously smaller files are cheaper/faster for everyone to download, and faster delivery means less time for things to fail and higher completion rates. Generally, we recommend using zip compression, as it's pretty much a de facto standard at this point (and is supported on Solaris OS as well). There are many free and open source distributions product teams can use, such as Info-ZIP, bzip2, and 7-Zip. You should experiment to see which one provides the best compression (in one test here, 7-Zip outperformed the others by about 20%, but your results may vary). There are also proprietary compression programs ("NOSSO," for example) that can greatly outperform "vanilla" zip compression, but they come with a price tag. If your download size really really matters, they may be worth investigating.

2) Use standard MIME types for file name extensions, for example "filename.zip" and "filename.exe." Web servers, browsers, download managers, and many other pieces of the Internet infrastructure rely on standard MIME types to ensure download transactions work correctly and consistently. So, don't release files without a MIME type extension (for ex., use "README.txt," not "README"), and don't make up file extensions or use non-standard ones, such as "filename.aa," "filename.bb," etc.

3) For large files, offer alternatives. Not everyone can successfully download a large file, especially ones larger than 2 GB. Other choices should be provided, such as a segmented ("chunked") version of the large file, CD images instead of a single DVD, and/or the option to order free or low cost CDs or DVDs. See my earlier post on "Breaking the Large File Barrier" for more details.

Soon after releasing our new download system, we received comments that the command line download client wget (and others) no longer worked. This is a result of a new session-based security model used to protect download links from being copied and re-used (which could circumvent numerous Legal requirements necessary to authorize a download). 

We have implemented a work-around now, so I just wanted to put this quick note out there to increase awareness of its availability. If you'd like to use wget or similar tools with Sun Download Center, please review the work-around, posted on our new Downloads FAQ Wiki.

Sorry for any inconvenience this has caused -- it was one of those unintended consequences. We believe we have a solution for it and will implement it as soon as we can. In the meantime, I'm pleased we could at least offer a temporary fix.

Get the work-around script here: Using wget with the Sun Download Center

OK, so things took a bit longer than we expected, and we hit a few bumps in the road, but our new download system is now fully operational and humming along! Just this week we finished moving all sun.com downloads onto it, and we're extremely pleased to reach this milestone.

To help introduce the new system (and also get newcomers and experienced users alike downloading as quickly, easily, and successfully as possible), please check out our new demo video. It features a walk through of the Solaris OS download experience, as well as helpful tips on using Sun Download Manager. The video takes less than five minutes.

I hope you enjoy the video, and more importantly, that you benefit from the enhancements we've made to downloading from Sun. As always, your suggestions for further improvement are welcome. You can leave me a comment, or if you encounter any questions or issues, please let our customer support team know. Thanks!

We've been working on a new internal wiki to capture download best practices gleaned over the past 11 years of designing and managing our high capacity download systems. I think this is good and valuable information so wanted to blog about some of our findings. This will be the first in a series of posts on the subject.

First, let's consider our goals in defining and publishing these practices. At a high level, I'd sum them up as follows:

• Ensure customers are able to download successfully, as fast as possible and with minimal hassle.
• Provide a consistent and efficient download experience.
• Increase completion rates and reduce abandonment.
• Ensure our systems and processes optimize the conversion of downloaders into (paying) customers.
• Save time and bandwidth (cost) for Sun and our customers.
• Within Sun, document the processes to release software for download, and ensure they are followed.

Our first best practice is seemingly simple yet frankly it's not been followed consistently over the years: use a unique file name for every file in every product and version release. There are several reasons why this is so important, the first of which is unique to our systems:

Sun Download Manager uses special "verification property files" (VPFs) to checksum files as they are downloaded from Sun Download Center. The VPFs are mapped to unique file names. So, here's where we get into really deep water if file names aren't unique from release to release! If we release a product with file name "product.zip", a VPF will be created called "product.zip.sdm.zip" that SDM grabs automatically and uses to checksum the file and ensure a successful download. Now if the product team releases the next version of the product but does not update the file name, a new VPF will be created with the same name as the prior one. This works fine for downloading the new product. But when users try to download the prior version, there will be a checksum mismatch between the old file and the new VPF -- the download will fail. And that's why this is so critical on our system.

Here are a couple of other good reasons (not unique to Sun's download systems):

• Downloads often flow through content distribution networks which cache files around the world for fastest delivery. Unique file names ensure caching works most efficiently and eliminates the potential for delivering incorrect content from the caches.
• Unique file names reduce potential errors when locating, copying, moving, updating, reporting on, and archiving content.
  

It's really not hard to do, so we hope you'll follow this practice!

I've been blogging for a while now, and I guess it was inevitable that wikis were in my near future as well! Having spent a lot of time and energy learning HTML and web publishing, I was hesitant to dive in and learn "wiki language" and the new publishing method. But a little encouragement from our VP goes a long way -- Curtis suggested we take a stab at a wiki to involve more of our community in the areas of download support and information.

So, we've taken the plunge and have posted a new Sun Downloads FAQ on wikis.sun.com. We already have a comprehensive FAQ devoted to Sun Download Center, and there's no point in repeating all the content in a wiki format. Rather, we want to broaden the scope to include all Sun downloads (open source products, for example) and related areas. I think we should be open to non-Sun content as well, if it relates to improving the download experience and helping our customers with any download questions or issues.

Kudos to Lori Holmes who got the site up and running and seeded the content with a few of the top questions from the SDLC FAQ. Next, I dutifully scanned the wiki manual and jumped in with my first contribution (including a shameless plug for my blog). I would love to add links to more expert resources around the web, so please let us know any suggestions.

I believe anyone who's a Sun employee has read/write privileges automatically, and your contributions are welcome. Those outside Sun will need to request edit permissions to contribute, but I'm sure we can figure out how to enable that if you'll let us know of your interest.

Time will tell if we can get critical mass around this effort, but at the least it allows us to quickly put out new information ourselves (without bugging our web publishing team to update the SDLC FAQ). Hopefully it will evolve into a valuable resource, ideally one with contributors far and wide!

"Large file" is actually a technical term for files larger than 2 GB (and to be really precise, that's 2³¹ or 2,147,483,648 bytes.) In the past, we've not been able to distribute files over 2 GB on Sun Download Center (SDLC). Our engineers tell me that's because 32 bit systems cannot handle signed integers greater than 2 GB. Way back when we built our "old" download system, it didn't really seem to matter, as we would never try to offer files that large for downloading (kind of like not worrying about Y2K back in the 1980's!). But times have changed, and with the proliferation of large, single file DVD ISO images that many Linux distros use, it's no longer uncommon. (Of course, this goes hand-in-hand with the proliferation of broadband access.)

As we built our new download application, large file support was a requirement. But, we were still stuck with a large file limit in some of the older code in Sun Download Manager (SDM). As using a download manager is really, really helpful for files this large, we appeared to be unable to proceed. We were very aware of this limit and had a high priority mandate to fix it, but we haven't had enough engineering resources to take that on yet. The pressure was building, however, as the Solaris OS team really wanted to be able to release single large file DVD images (and I don't blame them).

That's when things got interesting. Internally, we were testing our new download system, and we put a few large files on it. One of the testers sent in some test results saying, "Successfully downloaded the 3.2 GB test file using SDM." Impossible, I thought, there must be a mistake. But no, the tester insisted it worked. So I tried it myself -- it worked! They say "ignorance is bliss," and thankfully this tester was unaware of what we all knew "would not work" and simply went for it. It was quite a surprise.

Now these types of bugs really don't have a habit of fixing themselves, but we figured out what's going on.

For files on SDLC, we generate what we call "Verification Property Files" (VPFs) that contain the checksums SDM uses to checksum downloads real-time as they are received. Another piece of data in VPFs is the file size, and that's how we get around this limit in SDM. It turns out that as long as there is a VPF for a large file (and we create them automatically for all files released on SDLC), SDM can get the file size from the VPF and it all works! When there is no VPF, the file size is part of the header info sent from the web server, and this is when things break. (Some older web servers can't handle the large numbers either.)

So, bottom line, after a bunch more testing, we've just released the first ever single large file on SDLC -- the latest version of Solaris Express Developer Edition (~ 3.7 GB DVD ISO image). This is a small but significant milestone after years of butting up against the 2 GB limit.

Now the larger the file, the more that can go wrong, so if you give this a try, please do use SDM. And here are some notes and "best practices" we gleaned from rolling this out:

• The "32 bit" limit isn't unique to our systems but can affect servers, routers, operating systems, and clients throughout the network. For example, if a Windows XP system uses a FAT32 file system rather than NTFS, there's no way it's going to work -- the OS simply can't handle the file. (Thanks to openSUSE.org where I found that tip.)
• As a result, we highly recommend to our product teams that they not rely solely on a large file for distribution, as it's not going to work for all customers. Offer options such as a "chunked" version of the DVD that users concatenate after downloading in smaller pieces. Or offer multiple CD images instead of the DVD (as we do for Solaris). And finally, offer a hard media version (DVD) that users can order inexpensively (or better yet, free) and is then shipped to them.
• Use an up-to-date browser and fully patched, modern operating system to be sure large files are adequately supported on the client end.
• Absolutely do not attempt this with a slow line, like a dial-up modem. You can expect it to take about 40 hours per GB on a 56K modem.
• Use a download manager so you can resume where you left off in case anything goes wrong (you do not want to have to start over from the beginning). You can also pause and resume, if you're running out of time.
• Make sure you have at least twice the size of the file in free disk space -- with these large files, that's actually quite a bit of disk space. Operating systems typically make a temporary copy of the file while downloading, then copy it to its final location, so you must have the extra space.
• And a couple of notes specific to SDM:
• As noted previously, SDM will not support large files except on SDLC, so don't try it on other sites (until we can get this fixed).
• When SDM finishes downloading the large file, there is internal processing that must take place before the download is actually complete. Due to the huge file size, this processing can take several minutes. As a result, the SDM progress bar will say "100%" while the Status still says "Downloading data..." Be patient and do not close SDM. After a few minutes, the Status changes to "Downloaded", and the download is complete.

Hopefully this first large file release goes well and is the first of many. If you give it a try and have any problems or questions, please let us know -- the feedback is very helpful as we learn the ins and outs of large file distribution over the Internet.

After a few delays due to the complexity of the undertaking, I'm really happy to say we're back on track with the roll-out of the new download system that powers Sun Download Center. We are following a "phased product migration" plan which means we will not transfer all the products in one "big bang" from the old system to the new -- too much risk in that approach.

Back in October we put the first few products out but then had to hold pending some more back-end fixes. In early December, we released the fixes and then released 2000 of our lower volume downloads onto the new download system. A few days after that we released three high volume download products to slowly add load to the system. Among those is Sun Download Manager, which gets close to 30,000 downloads a month. So far so good -- all is working smoothly, and we have a ton of capacity remaining on our new servers.

Most of Sun closes for a week between Christmas and New Years, and so we decided to wait until after the break to move the remainder of the downloads onto the new system, including our "crown jewels" such as Solaris Operating System, Sun's Software Portfolio, Java software and developer tools, etc. This ensures that we'll have the appropriate support and engineering staff available during the transition.

As you download from Sun for the next few weeks, you'll find some products on the old system and some on the new. Other than a few UI changes and other mostly-subtle improvements, this should be transparent and not an issue for any of our customers. (You can tell you're on the new system because the URLs start with https://cds.sun.com/...)

I'll be off shortly for a couple of weeks of much needed rest and relaxation so wanted to send my best wishes to you all for the holidays and a great 2008!

In a world of unintended consequences, one I often think about is not realizing how many new friends I would make because of my kids. Through numerous events, car-pooling, baby sitting, play dates, parties, and years of schooling with the same kids in their classes, I gained a whole new set of good friends (the parents that is, not the kids!).

Similarly, I hadn't realized when starting my blog that you can make some very interesting connections and virtually meet people who share your interests. It's a great benefit, and here's a great example. Soon after starting my blog, I "met" Anthony Bryan via some thoughtful, intelligent comments he left on subjects I was discussing. He must've found me due to my interest in ESD and download managers -- if you look around, there's simply not that much written about those subjects. And it's always great to find others who share your interests and passions.

I first mentioned Anthony's project, Metalink, almost two years ago, when he was just starting to gain traction. We've kept in touch, and it's really amazing to see how it's taken off since then. It's no accident of course -- it took perseverance, in combination with his clever, well-implemented, open technology. Metalink filled a gap in download managers and systems, providing for much needed enhanced redundancy, load sharing, and fault tolerance for large file downloads.

There's a long list of products now that incorporate Metalink, a sure sign of growing acceptance and success. I was going to mention a few, but I see his home page is up-to-date and says it much better than I can, so take a look. Also, here's an informative interview with Anthony about his project and its benefits.

So, what about Sun Download Manager (SDM), does it use Metalink? Well, no, not yet at least. The main reason is that SDM's primary audience is customers downloading Sun software from Sun Download Center (SDLC). Access to this software is carefully controlled for security and export control reasons. We use load management to distribute the load on multiple servers in our own data centers. As there aren't mirrors our there for this class of software (i.e., mostly not Sun's open source software), we lose one of the main advantages of Metalink. That said, we do know a lot of people use SDM on other sites because it's a good, simple, free, cross-platform download manager. So that sounds like a good argument to build in Metalink support in the future! (I'll say it before Anthony does.) I'll certainly keep it on the radar, but must admit all our engineers are tied up finishing our new download system at the moment.

In the meantime, I see that Metalink is used for OpenOffice distribution and was further pleased to see it mentioned in a number of other Sun blogs. 

Congratulations Anthony, and I hope Metalink is just the first of many successes for you.