Sun SPOT Feeds
Author: maehem
Posted: Fri Sep 05, 2008 4:49 pm (GMT -7)
Topic Replies: 1

I just tried this out and it works for me.

Platform: Linux Ubuntu 8.04
Eagle: 5.2.0 light(free version)

checked out with:
Code:
svn checkout https://spots-ebones.dev.java.net/svn/spots-ebones/trunk spots-ebones --username <myusername>


Started Eagle.
File->Open->Schematic (browse to file).
Just loaded up and looked OK.

Tried both 2-layer files and 4-layer files.

Make sure your version of SVN is up to date.

Also, what platform are you on? (Windows, Mac, Linux)?

It's a big day here ...


We have released the latest updates to a couple of the NetBeans Sun SPOT Modules which include a few minute Manifest File bug fixes, and the latest Beta (which is likely to be the final release) of the Blue (v4.0) Sun SPOT SDK.


But for me, the biggest deal is that I was just cleared to release the v4.0 version of the Sun SPOT Manager Tool. Tere are some significant additions to this version, especially for first-time SDK users. The biggest news is that there is no more (and I mean NO MORE!) configuration to go through. Just run the Sun SPOT Manager tool. If you're missing anything, like the Sun JDK, or Ant, or NetBeans, the Sun SPOT Manager will install it for you! And even better, it will configure it for you!


What's that mean? Well, for those of you on Windows, there is no more editing the Registry to get your JAVA_HOME and ANT_HOME and PATH variables set up correctly. I do that for you. There is no more hunting around for the NetBeans Sun SPOT Modules. I install those for you. For an easy install, just "follow along" and accept all the defaults, and all the suggestions. In about 20 minutes, and after one reboot, your Windows XP box will be up and running the Sun SPOT SDK.


For you Vista users, there's only ONE additional step: Start the Sun SPOT Manager Tool, but as soon as it starts, and creates a desktop shortcut for you, quit it. Right click on the desktop icon of the Sun SPOT Manager Tool, and select "Properties." Go to the 'Compatibility' Tab, and click 'Run As Administrator.' Then restart it by double clicking on it. Just follow along, and in about 20 minutes, and one reboot, you'll be up and running.


Linux user? Same thing. Just run the Sun SPOT Manager tool, follow along, and when prompted to do so, log out, then login again.


Done!


I'll provide more details soon, but I wanted to get this posted before the Tropical Storms and Hurricanes arrive.


[ It wasn't that she had a rose in her teeth, exactly. It was more like the rose and the teeth were in the same glass. ]

There are some pictures here: [link]

-r

On Sep 2, 1:53 pm, "Tennessee Carmel-Veilleux"

Hi everyone,
It has been a while since anything was said about this project. I will have
more free time in October and I want to make a website about EDGAR1, as well
as make a proper open-source software and hardware project on Java.net. We
already have the project page, now we need to fill-in the blanks (change the

So I’ve blogged about using the Precon Temperature/Humidity sensor before, but it is time, apparently,to re-visit the topic. We’re working on a project with Vodafone, and I needed to write a brief “How To” for them, so I thought I’d just make it here so everyone (both of you) could benefit.


First, this sensor is possibly the easiest to hook up and interact with of any I have worked with. It has a simple analog temperature sensor and an analog humidity sensor that are just read as a percentage of the reference voltage supplied to the unit. So if the voltage supplied is 3.3v, and the humidity value read is 3.3, then you’re at 100% humidity. And it’s a simple linear scale, so there is no complex conversion, etc. The same is true of the temperature value, though the scale is -30° C to +100° C, so you have to do a little bit of adjusting of the raw value.


The wiring is extremely simple:



Precon1.png


That’s Pin 1 — Temperature — of the Precon to Pin A1 on the Sun SPOT for Temperature readings. Pin 2 of the Precon to +3v on the Sun SPOT — to power the sensor. Pin 3 — Relative Humidity — of the Precon to pin A2 of the Sun SPOT for RH readings. And Pin 4 — Ground — of the Precon to Ground on the Sun SPOT. The only other connection you need is +3.3v on the Sun SPOT to Pin A0 on the Sun SPOT for reference Voltage (vref). Since the sensor readings are a percentage of supplied voltage, you have to keep track of the supplied voltage, and this is how.


That’s it for wiring. Really.


Now, the code is just as simple. Once gain, really.


I used NetBeans, of course, to create the project using the Sun SPOT Application Template Module (for information of installing these templates, see here). Here’s the code for the main class:



/*
* RHGPS.java
*
* Created on Aug 21, 2008 4:54:22 PM;
*/

package org.sunspotworld;

import com.sun.spot.peripheral.Spot;
import com.sun.spot.sensorboard.EDemoBoard;
import com.sun.spot.sensorboard.peripheral.ISwitch;
import com.sun.spot.sensorboard.peripheral.ITriColorLED;
import com.sun.spot.sensorboard.io.IScalarInput;
import com.sun.spot.util.*;
import java.io.*;
import javax.microedition.midlet.MIDlet;
import javax.microedition.midlet.MIDletStateChangeException;

/**
* The startApp method of this class is called by the VM to start the
* application.
*
* The manifest specifies this class as MIDlet-1, which means it will
* be selected for execution.
*/

public class RHGPS extends MIDlet {

private ITriColorLED flash = EDemoBoard.getInstance().getLEDs()[0];
private static int SLEEP_INT = 250;

protected void startApp() throws MIDletStateChangeException {
System.out.println("Hello, world");
new BootloaderListener().start(); // monitor the USB (if connected) and recognize commands from host
IScalarInput vref = EDemoBoard.getInstance().getScalarInputs()[EDemoBoard.A0];
RHSensor rh = new RHSensor(EDemoBoard.A1, vref);
TempSensor ts = new TempSensor(EDemoBoard.A2, vref);
long ourAddr = Spot.getInstance().getRadioPolicyManager().getIEEEAddress();
System.out.println("Our radio address = " + IEEEAddress.toDottedHex(ourAddr));

ISwitch sw1 = EDemoBoard.getInstance().getSwitches()[EDemoBoard.SW1];
flash.setRGB(100,0,0); // set color to moderate red
while (sw1.isOpen()) { // done when switch is pressed
try {
// done when switch is pressed
flash.setOn(!flash.isOn());// Blink LED
System.out.println("Temp: " + ts.getTemp() + " Degrees C.");
System.out.println("Humidity: " + rh.getRH() + "%");
Utils.sleep(SLEEP_INT); // wait 1 second
} catch (IOException ex) {
ex.printStackTrace();
}
}
notifyDestroyed(); // cause the MIDlet to exit
}

protected void pauseApp() {
// This is not currently called by the Squawk VM
}

/**
* Called if the MIDlet is terminated by the system.
* I.e. if startApp throws any exception other than MIDletStateChangeException,
* if the isolate running the MIDlet is killed with Isolate.exit(), or
* if VM.stopVM() is called.
*
* It is not called if MIDlet.notifyDestroyed() was called.
*
* @param unconditional If true when this method is called, the MIDlet must
* cleanup and release all resources. If false the MIDlet may throw
* MIDletStateChangeException to indicate it does not want to be destroyed
* at this time.
*/

protected void destroyApp(boolean unconditional) throws MIDletStateChangeException {
for (int i = 0; i < 8; i++) {
flash.setOff();
}
}
}


So that's the Main portion. You will notice that there is reference to 2 other classes, RHSensor and TempSensor. Those are amazingly simple classes, listed here:



/*
* To change this template, choose Tools | Templates
* and open the template in the editor.
*/

package org.sunspotworld;

import com.sun.spot.sensorboard.EDemoBoard;
import com.sun.spot.sensorboard.io.IScalarInput;
import java.io.IOException;

/**
*
* @author davidgs
*/

public class TempSensor {

private IScalarInput temp;
private IScalarInput vref;

/**
*
* @param pin Pin number for analog reading of Temperature
* @param vref IScalarInput pin for voltage reference
*/

public TempSensor(int pin, IScalarInput vref) {
this.temp = EDemoBoard.getInstance().getScalarInputs()[pin];
this.vref = vref;
}

/**
*
* @return Double value of degrees C
* @throws java.io.IOException
*/

public double getTemp() throws IOException {
return (((double) temp.getValue() - 115) / (double) vref.getValue()) * 100;

}
}




/*
* To change this template, choose Tools | Templates
* and open the template in the editor.
*/

package org.sunspotworld;

import com.sun.spot.sensorboard.EDemoBoard;
import com.sun.spot.sensorboard.io.IScalarInput;
import java.io.IOException;

/**
*
* @author davidgs
*/

public class RHSensor {

private IScalarInput rh;
private IScalarInput vref;

/**
*
* @param pin pin number for the relative Humidity sensor
* @param vref IScalarInput pin for voltage reference
*/

public RHSensor(int pin, IScalarInput vref){
this.rh = EDemoBoard.getInstance().getScalarInputs()[pin];
this.vref = vref;
}

/**
*
* @return double-precision relative humidity as calculated by:
* rh pin value / vref pin value * 100
* @throws java.io.IOException
*/

public double getRH() throws IOException{
return ((double)this.rh.getValue() / (double)vref.getValue()) *100;
}

}

And that's all there is to it. A simple ant deploy run will put the application in your Sun SPOT, and start it spitting out Humidity and temperature readings every 1/4 second.

I hate to use public humiliation as a means to get action, but sometimes, it's the only way. I've been trying for a week to get some response -- any response -- out of Facebook support. I know, you may think that is an oxymoron right there, and given my current experience, I'm inclined to agree with you.


How hard is it for a support organization to actually respond to issues submitted in a finite amount of time? How hard can it really be? Seriously.


Here's the issue. I own a group on Facebook for the Wake County Special Education PTA (Wake SEPTA). I'm their de facto webmaster. So I decided to promote them on Facebook. Why not, right? Well, I managed to somehow click the wrong button and change my location. Duh. Silly mistake. But it, apparently has dire consequences. I lost Administrative rights to my Raleigh/Durham based group, WakeSEPTA. This wouldn't be a problem if there were other Admins that could re-authorize me. But there aren't, so the group is 'orphaned' such that no one can join, and I can't get back in to administer it.


So last week I began asking Facebook Support for help. So far, nothing. Not a peep. Not a response. No help. Nothing. Is the help alias hooked straight to /dev/null? Sure seems like it.


One would think that Facebook would be interested in actually supporting users. But apparently once you're hooked, they don't much care. I'd love to get this issue resolved. I would have loved to have gotten this issue resolved a week ago. Seems like an easy thing to fix. But it does take some response from Support before it can get fixed.


[ Wethern's Law:


Assumption is the mother of all screw-ups. ]

Rocket designed, made and fired by me. It was designed to house an accelerometer (Sunspot device) to be able to obtain data from a rocket take off. Rocket name: S.S. Sarah Location: School oval Eninge: Estes C6-3 Weight: 150g

Author: blake14321
Keywords: accelometer acceleration sunspot device smoke lift off rocket prototype spaughts home made design build engine c6-3 estes aerotech flight
Added: August 19, 2008

So my team here has been working on a framework for data collection called Yggdrasil lately. Which has kept us all very busy, one of the milestones we are coming up to involves working with Warren Wilson college on an installation of sensors that they have been working with in the Panamanian Rainforest


. 2193499188_e642324cc0.jpg


pic stolen from blog link below -ed


In preparing for it we've started cleaning up parts of Yggi for external consumption, if your interested in data collection frameworks ( you know you want to, all the cool kids are doing it ) then check out chris' blog and a quick example


[Warning: example is pre-alpha, no really it is, really pre-alpha]

Last week I gave a tech talk at Google about Sun SPOTs. It was fun. Thanks to my friend Will Rusch for inviting me. The venue was a little noisy and distracting, plus the Semaphorebot wasn't working for some reason, but, if you have an hour and want to learn about Sun SPOTs give it a watch.



Of course, this and other Sun SPOT videos are tagged with "spaughts"

just testing that my month long transition to a new laptop has kept my blog-tool intact

As I post this, it looks like at least one balloon is in the air.
[link]

Google Tech Talks July, 25 2008 ABSTRACT Roger will present a quick overview of Sun Labs, the research arm of Sun Microsystems Inc. He will then talk about Project Sun SPOT (http://www.sunspotworld.com), a completely open source development platform designed to inspire the next generation of gadgets. Sun SPOT devices are Java-based, small, wireless, mesh networked, secure, sensors and effectors that can be used in everything from sensing the environment to controlling robots. He will show how Sun SPOTs are being used by our community of developers to make pets do tricks and make bicycles fly. More info at http://www.sunspotworld.com and open source at http://spots.dev.java.net. Speaker: Roger Meike Roger is Senior Director, Area 51 and Director of Operations, Sun Microsystems Laboratories. His background is in cognitive science and his career has led him back and forth between new start companies and large research organizations. While his background is mostly in software, he also enjoys consorting with hardware folks. He has been accused of being many things including photo enthusiast, sailor, ham radio operator, musician and techno-geek/nerd.

Author: googletechtalks
Keywords: google techtalks techtalk engedu talk talks googletechtalks education spaughts
Added: July 29, 2008

One of the exciting parts of our work here in Sun Labs is that more and more of what we do is done in the open. It gives researchers a special thrill when they get to see people using their work and taking it beyond what they planned. The Sun SPOT community has been growing impressively recently. We have been seeing more and more work done around Sun SPOTs and it is great. Part of a healthy community is building an ecosystem. As a research group, its always difficult for us to serve both our research goals and provide a market opportunity for others.20070422_TrackBotBetterCable_0028.jpg


One of our first was with Systronix who built the Trackbot. They built a complete robotic platform which can use a Sun SPOT for its brains. It's a nice experimental platform that can take advantage of some of the interesting features of the Sun SPOT like Java-based development, wireless communication for untethered development and swarm behavior. It allows you to build a variety of mobile applications that can be quite compelling. The Trackbot has been available from Systronix for some time now and has a loyal community of followers.


200807280942.jpg 200807280943.jpg

A more recent entry to the Sun SPOT commercial space is the recent announcement of the availability of eProtoBoards from Brilldea.com. We recently open sourced this board, but as a research group simply have not been able to build and distribute another product at this time. The net effect was that many people in the community wanted the board and our open source efforts were just teasing them. Well, the good news is that Brilldea.com stepped in to make the boards available. This is a great step forward for the community. We want Sun SPOTs to be a great platform for innovation and experimentation. The eProtoBoard is a big part of that. Now they are only selling the bare board, but for those who are designing their own circuits anyway, having to put on the few extra parts should not slow anyone down.

Of course hardware is not very exciting without software and I'm happy to say that the software innovation also appears to be going along quite well. One of the most interesting recent developments here is the announcement of a new radio stack from the folks at University of Karlesruhe in Germany. They did some experiments and found that our radio stack was not to their liking, so they wrote their own. We were hoping to encourage innovation in this area since we think it is ripe for a variety of improvements and optimizations for particular application areas. We hope to see a lot of experimental radio stacks pushing different approaches to distributed communication. Of course, one great thing about the KSN RadioStack is that it is all open source. You can go to their website today and take a look at their YouTube video below.

In this video, the self-healing capabilities of the KSN RadioStack are demonstrated using Sun SPOTs For more information visit: http://www.ipd.uni-karlsruhe.de/~bestehor/KSN/RadioStack

Author: deryomega
Keywords: SunSPOTs spaughts network stack
Added: July 27, 2008

Project from S.Lamora and D.Valles, students of the Faculty of Informatics at the Universitat Politecnica de Catalunya (UPC). The project is a tracking system of mobile devices. It is a real time application to know the mobile's position in a specific moment. Actually, the result is an area, more or less precise, where is the mobile.

Author: mjblesa
Keywords: spaughts SunSpots REPPRO FIB UPC
Added: July 21, 2008

Yggdrasil - A Data Collection Framework

Trying to explain how a Sun SPOT works to a native Spanish speaker was proving more difficult than I had anticipated.  I had been attempting to explain to our guide, Joel, why I was standing in a clearing holding a blinking piece of hardware and looking pensive.  I had been trying my best to communicate using broken Spanish (which in truth, was mostly slow English) the concept of mesh routing.  Eventually I reverted to the what I thought must be the universal sign for wireless data transfer; holding the SPOT in one hand while wiggling the fingers of the other and moving the hands apart.  I may have confused the matter later in my explanation when I used the universal sign for rain (wiggling the fingers of one hand and dropping down to the SPOT) to indicate the Otter boxes we were using to keep the SPOTS safe were weather proof.



 Although I do not feel I was successful at breaking the language barrier during my trip to the Cocobolo Nature Reserve in Panama, we did do some pretty cool stuff with Sun SPOTS.  I was with my professor, David Abernathy who had co - written a grant to, among other things, install a wireless sensor network on the reserve.  We were bent on setting up a solar powered Sun SPOT, bouncing the data back to the thatch hut, and uploading it via satellite for visualization on Google Earth.  With us, we brought a special weather proof Sun SPOT.  The device had been constructed by Lawson Revan, my friend and neighbor at Warren Wilson College.  He had cleverly mounted a Sun SPOT inside an Otter box, and routed wire from the Vext port through a 5 volt regulator and to a cluster of solar panels harvested from garden lamps and mounted on a refrigerator panel.  The device performed very well under the Panamanian sun, outputting nearly 5 volts and up to 150 mA's.


I had written a simple Sun SPOT application to gather temperature, light, and power data and stream it back to the host.  From there, it was logged to MySQL.  The data was accessible via a simple Apache web server that interfaced the data with Google Earth.  On our second day in the Jungle, we had real time data streaming via satellite to the internet.


Since then the Sun SPOT data collection project has grown rapidly.  The Cocobolo Nature Reserve is one of several projects that seek to use SPOTS as data collection tools.  It is for this reason that Yggdrasil was developed, a data collection framework for Sun SPOTS.  The framework is currently in pre - beta, and is able to take sensor readings at a given time interval, relay the data back to the host, and interface with a database.  Throughout the process the Sun SPOT sleeps when it is not working, ensuring that no power is wasted.  The project is moving forward quickly.  Lachlan is working on the database component, John is building a 3D Node Browser, Robert is improving the underlying network code, and I am tying together a Configuration Manager.  We are planning a working deployment in Panama in late summer / early fall.

As a whole, our project is part of a growing cluster of projects that seek to collect data and store it.  Because of this, more and more data is becoming available on the cloud.  WIRED magazine has coined this the Petabyte age, in which scientific theory is taking a back seat to the investigation, interpretation, and visualization of vast quantities of data.  Technologies that make data collection and storage easy and rapid are changing the face of science.  Historically, researchers have hypothesized, collected data, and determined what it meant.  However with this new influx of information, we can design studies around the data we already have, or data that we expect to have.  In fact, we may find ourselves crunching data purely for the sake of crunching, and finding things we were never looking for in the first place.

As the cloud grows in size, and the speed at which it grows increases, the opportunity to find important trends in data grows exponentially.  It is essential that the scientific community not only be be equipped with the best tools technology has to offer, but also be free of technical problems so that they can spend their time exploring data, not debugging code.  This is a time in which our ecosystem is changing rapidly, for known and unknown reasons.  Data collection, storage, and interpretation is no longer a luxury, but a crisis discipline.  If we are indeed flapping the wings of a flying machine that was never meant to fly, as is postulated in Ishmael, now is our chance to rebuild in flight and ensure we make it to our destination safely.

So back in 1998, while I was the Technology Manager for Sun Ray, I set a goal for myself of winning the Chairman's Award (as it was called at the time). This is a very prestigious award at Sun, and is remarkably hard to win. I worked myself near to death for years trying to win this award, to no avail.


3 years ago I left the Honeycomb Team in the Labs a few months before they won the award (now called the Sun Innovation Award). Once again, close, but no cigar.


So I'm very happy, and feel honored, that the Sun SPOTs Team has won an Innovation Award for FY08. This is a very high honor indeed, and I am pleased to be a part of such a fantastic team that has produced an innovation worthy of such high honor. This was certainly a Team effort, and everyone associated with the project deserves the recognition. So, my hat is off to my team members for getting me somewhere I have aspired to for many years.


Team Members:



Again, this was a Team effort of enormous proportions. The Hardware folks delivered unmatched quality with a stunningly low level of returns and errors. The Software team (Squawk and the Sun SPOT Team) delivered some remarkably stable software, and we have seen the adoption of Sun SPOTs increase through both sales and our Sun SPOT Open Grant Program.


But without our users, customers, champions and supporters, we'd be toiling in obscurity, so it is really thanks to all who use Sun SPOTs, are enthusiastic about Sun SPOT, and the entire community. Thank you.


[ I'm rated PG-34!! ]

As of 2 minutes ago, the search for Nancy Cooper, our neighbor, is officially over. Her body was found last night. I don't even know what to say. I am at a loss for words, and am deeply saddened. The neighborhood of Lochmere is an open, friendly, and close-knit community and this hits hard.

My son is enrolled in a summer science camp that he is enjoying quite a lot.

Last night he brought home a small trebuchet that he made in class. It was pretty nice and once we determined that he could throw an AA battery the quite a way in the back yard. He asked me the question, "I wonder how many Gs that battery gets when its thrown?" Well, dad's live for these kind of questions, especially when they have a set of Sun SPOTs in the house. So we got out the Sun SPOTs started throwing them around the yard. The telemetry application that comes with the Sun SPOT kit is ideal for this project. It provides a live feed of the accelerometer data from the Sun SPOT. That gave us live data as the SPOTs flew around the yard. Its a pretty small trebuchet, so it couldn't throw anything as heavy as a Sun SPOT very far, but it was interesting to see that the acceleration as the Sun SPOT was dragged along the platform was only about 3Gs, while once it started its real swing it was accelerated right up beyond 6 Gs. Then in the throw graphed below (displaying only y and total acceleration), you can see that it spun in the air 5 complete revolutions before landing unceremoniously with a >6G shock and bouncing in the grass several times then finally coming to rest. It was pretty fun.

200807142201.jpg L1010642 L1010641

We just had a short time before dinner, so we didn't try to calculate how high it went in the air or anything tricky like that, but I see no reason not to since you can tell when the trebuchet let go of the Sun SPOT and when it landed. You can calculate the acceleration involved and determine, to some approximation, how high it went.. I think this type of project could make for a fun grade school science project.

I've had ideas for other Sun SPOT related grade school projects like the SPOT-drop. Its like an egg drop contest where kids design enclosures for the eggs and then drop them from a reasonable height onto a hard surface. If the egg breaks they lose, if it doesn't, they win. Well, if you substitute a Sun SPOT for the egg, you can give the students actual data accelerometer data about how well their enclosure did rather than a binary break/no break indicator. If you have real data you can get into a lot more subtleties of what designs are best at protecting their cargo. Anybody want to give it a try and tell me how it goes?

This is pretty cool. A spider bot controlled by a Sun SPOT. Nice job marvelouskobe!


I found this on YouTube thanks to the "spaughts" tag that users put on videos and pictures.

Remote-controlling a spider robot equipped with a Sun SPOT

Author: marvelouskobe
Keywords: sunspot spaughts
Added: July 14, 2008

There's now a site dedicated to the search for Nancy Cooper. If you can volunteer, please do so! http://nancycooper.blogspot.com/

Family and friends of Nancy Cooper thank all of those who helped on Sunday with the search for Nancy. They had 150+ people looking on trails and covering the city with close to 2000 fliers. Truly the fliers are everywhere, which is a good thing.


The plan for today (Monday) is to go out and search more. There will be headquarters at both Lifetime Fitness and Java Jive starting at 9 am. From there people will be sent to a designated area to search in larger groups focusing on the area 20 feet to the sides of the trails and also out-lying neighborhoods. Volunteers are needed throughout the day.


Needless to say, we'll be out searching today. Hope you can join in.

Just a quick note that the team was working on some long wifi links, check out


200808061126.jpg


Roberts pic of me up on an antenna mast. just another day at sun labs :)

Copied verbatim from Rich Sharple's blog, but this is big ... and scary for us.


One of our neighbors - Nancy Cooper has been missing since the morning of Saturday, July 12. She went for a run around 7am on Saturday morning in theLochmere / Regency Lake area of Cary, NC and never returned. Nancy is white, has shoulder length brown hair and hazel eyes, she’s 5 feet 9 inches tall and weighs about 120 pounds. She was last seen wearing a T-shirt, running shorts and light blue running shoes.



There’s more information and photos on WRAL.com and the News & Observer. If you’re a local blogger - please help spread the word and If you have any information contact Cary Police at 919-469-4016 / 4012.

The Ravenswood site is several miles away from the Labs which is too far for a single patch of SPOTs so we need a way of sending and receiving data from the site. The picture below shows the distance between the Labs and Ravenswood site which is next to the bridge in the distance. DSC02673 The easiest and cheapest way of getting data to and from the site would be wifi. The only problem with using wifi would be the need to put repeater stations in the ground every so often because a single access point won't reach. However, the Labs have bought a couple of Ubiquiti Network's PowerStation 2 to boost the wifi signal so that we can run a wifi network from the Labs to Ravenswood. On Friday we tested the link between the Labs and one of the Director's houses over 8 miles away and got about 4Mb/s without any major tweaking. DSC02692

I am a avid digital photography enthusiast. I have over 100,000 pictures in my library. With my current digital SLR camera, a single raw image can be 14MB, and I can generate them at 5 images per second. I've done a few HD video editing projects, and as you can imagine, those also eat gigabytes of disk space like it's popcorn. Managing that much data causes significant issues for me as the household's resident IT manager. I have to keep it all backed up and available for my family. We have several terabytes of spinning rust in our home. Recently, as we left on a week long vacation, our neighbor asked, "if these California wildfires find our neighborhood while you are gone, is there anything that you want me to save." Given that he probably can't save the piano, it was easy for me to ask him to grab the backup storage device, with years of photos and records on it.

Of course, I also work with sensors, devices who's main purpose in life is to generate data. We are generating data at an incredible rate. All that data needs to be stored, so storage becomes more and more important. That's why Jonnathan is blogging about it and why we are announcing a slew of new storage products. Well, we at labs wouldn't want to be left out. That is why I'm happy to announce the availability of Celeste, an open source research project aimed at reliable, secure storage made of unreliable, non-secure parts, that scales beyond imagination.


200807101534.jpg

Celeste is a very interesting set of technologies. It is a distributed object store. That means that when you store an object in a Celeste system, it is split up into chunks that are stored on various machines spread across a network. The system hands back a handle, which later can be used to reconstruct the object. Because these chunks of data can be replicated or encoded for redundancy, the storage can be very reliable. This also has interesting implications for security, since each participating node knows only about the chunk of data it is asked to store. It does not know who stored it or what larger object that chunk of data belongs to. The system is designed to handle unreliable or even rogue nodes in the system. In addition to checking the reliability of the data as it is retrieved, nodes can keep track of the reputation of other nodes in the system. This means that when you save an object, you can ask to save it someplace VERY reliable or someplace with a good reputation for fast access. The society of nodes in the Celeste system use the reputations that are built up over time to match with the users wishes when storing future objects.

200807101543.jpg

The system is alive. When left alone it will continually monitor itself to make sure that the proper levels of redundancy are maintained as nodes come and go from the system. This has the effect of sloshing data around the system to level storage over time. This has an additional very interesting side-effect. Say you have a data center and you decide to buy one of our cool new storage boxes. If you could plug it in and tell it to start participating the the Celeste system, data would naturally start flowing into the new storage units. If you have some old storage boxes that you would like to retire, perhaps because they just draw too much power for the amount of data they hold, you can just turn them off. When you do this, the Celeste system will notice that some data has disappeared and perhaps the redundancy levels have fallen below the required threshold, so Celeste will start creating redundancy coded chucks of data, most probably in your new storage system. This means that your data will automatically migrate to the new storage hardware just by connecting the new storage and turning off an old one.

The heart of the system is the Distributed Object Locator (DOLR). This is based on a Distributed Hash Table (DHT), which is a very slick piece of technology. It allows a completely masterless set of nodes to work together to find any piece of data in linear time. This system as implemented in Celeste can scale incredibly. It can address yottabytes of information. Yes yottabytes - now you can tell your friends that you learned a new word. kilo, mega, giga, terra, peta, exa, zetta, yotta. A yottabyte is 1,000,000,000,000,000,000,000 (1000^8) bytes. That should be able to store all my photos with no problems.

Another interesting aspect of the system is that it is designed to be able to handle drastic failures. For example, suppose that you have a Celeste system that consists of 1000s of nodes spread across the United States. If someone digging a trench in St Louis, severs your main OC-193 cable between the east and west sides of the country, the system will continue to operate as two separate Celeste systems. Stores and retrieves can go ahead as normal. Eventually, when the damage is repaired, the system will stitch itself back together including flagging potential conflicts where two independent writes have been carried out on a single object. Additionally, a side effect of the way that the data is looked up in the DOLR is that data naturally migrates closer to where it is being used. This means that if you are accessing a lot of data and then travel half way around the world, the data can follow you to ensure speedy access from anywhere. In interesting side effect of these two behaviors is that one can imagine a scenario where your laptop is participating in a Celeste system. As you use data, it finds its way to your laptop. At the end of the day, you take your laptop home and work disconnected from the network. You read and write data locally on your hard drive. The next day you come in and re connect to the Celeste system and your data is seamlessly reintegrated.

A key differentiator between our system and other distributed storage systems is that Celeste is mutable. These distributed storage systems go to great length to never forget anything, so often they are designed to NEVER forget anything. In many cases this is not appropriate for real world use. Nothing is ever deleted and storage is never recovered, so these systems just grow and grow. We have many customers who wrestle with regulations that require them to keep customer records for a certain length of time and then guarantee that they will get rid of them. Celeste includes some secret sauce that allows it to really forget data. You can delete data, change it, etc. just as you would on a standard file system.

There are many other aspects like interesting security modes and possible modes of disconnected operation. This system is a research tool and not yet part of any product, so don't expect to just plug it in and replace you existing file system, however, if you are interested in this type of research I encourage you to check it out. The project is entirely open source.

You can learn more about the Celeste project here: http://www.opensolaris.org/os/project/celeste/

Congratulations to Glenn and Glenn for getting this great technology out to the world!

Hello again. In my last post I briefly introduced myself, now I'd like to take the time to explain in more detail what I'll be doing during my internship. Amongst other things, I'll be working with the SPOT team on the Yggdrasil Data Collection Framework. This framework is an effort within the SPOT team to create a programming framework that enables people with various levels of programming experience to easily create end-to-end applications i.e. all the code from visualising and storing the data on to the code that runs on the SPOTs. I'll be working on the SPOT side tools i.e. the code that runs on the SPOTs. This code will hopefully be used in SPOT deployments at the Cocobolo Nature Reserve in Panama and at Ravenswood in Menlo Park, CA as part of the salt marsh reclamation project. I'll be posting more about my work as its completed.

As some (many? both?) of you know, we have been running a special program for Education with Sun SPOTs. We have been accepting proposals, and granting Sun SPOT kits, for course development in Wireless Sensor Networks, Robotics, and a wide range of other topics. The courses developed through this program are required to be open source, and we're starting to get results!


RTI has submitted an interesting website with some of their course development activities and results. Check it out!


I'll post more as they roll in. If you're working on one of these projects, and you want to see your project listed here, just send me the link (again, if you already did and I ignored it) and I'll get it posted here!


[ The Kennedy Constant:


Don't get mad -- get even. ]

I think we (Sun) 'get it' and get it right, most of the time. Not on everything, for certain, but when it comes to communication, openness and transparency, we have that down. As a counter example ...


I use a nifty piece of software from BareBones Software called Yojimbo (I know, the name sort of blows, but it's a great 'digital junk drawer' for my Mac and helps me save, organize and keep track of lots of stuff in one place. But I digress). Don't get me wrong here, I love the product, and use it almost every day. But their communication strategy seems to be modeled on Apple's. Tight-lipped and reticent. The users on the Yjimbo users' alias are clamoring for answers, for insight into what may, or may not, be going to happen with the product, and where things are headed. the response from BareBones?


Yep, pretty much. Well, other than to point to their FAQ, which says:



We'd like to offer an iPhone version of Yojimbo, but can't currently say if/when this may happen nor what its feature set might be. As soon as we're able to provide more info about our plans, we will do so.



The equivalent of "we can neither confirm nor deny ..." and a non-answer. Don't get me wrong, I understand that at times it can be advantageous to carefully control the release of information so as to not over-promise, under-deliver, or to make a big splash with your announcements. But there is a lot to be said for involving your users, for making them feel involved, a part of the process, and invested in your products. A user that has a mere affinity for a product is not nearly as good as a user that has an investment in, and a commitment to, a product or company. Involving your users gives them the investment and commitment to make them useful assets, not just the other end of the payment stream.


And what has all this to do with Sun? Well, I really think that this little experiment we call blogs.sun.com is a shining example of what can come of a company that values open communication with its customers -- and potential customers. Very few of our customers, I would venture, are unclear about what we are working on, where we are investing, and what they can expect from us. Yes, we all want to see more from the stock price, etc. but it's pretty clear that we have a real commitment to open communication with our customers.


I wish more companies would show the same.


[ "Now is the time for all good men to come to."


-- Walt Kelly ]