Thoughts on Amazon Cloud Drive’s New Sync Client

I’m a big fan of Amazon. They sell me many things, and I am totally hooked on Amazon Prime. So my first reaction to their new new sync client for Cloud Drive was the same as many others, “What took them so long?” Then I wanted more details on how it works. From my experience as product manager for Mozy Stash, I believe that the efficiency of a file sync client makes or breaks the core offering. Delving in, this initial release of Cloud Drive is disappointing.

Size Matters

File sync clients must “just work.” One aspect for that is their demand on system resources.

Installer Package Size (Mac OS)

Let’s start by comparing the size of the installer package, which gives us a rough idea of how efficient the developers’ code is. The smaller the package, the more efficient.

• EMC Syncplicity: 11.7 MB
• Mozy Stash: 10.8 MB
• Google Drive: 24.7 MB
• Dropbox: 27.4 MB
• Amazon Cloud Drive: 16.6 MB

Amazon fares pretty well against the two most popular sync tools, but file sync is a hard game. Companies that venture into file sync typically have to learn numerous gotchas of sync, and their software expands as the developers make fixes to handle all of the intricacies. As Amazon does so–and adds more features–expect their package to grow accordingly.

Next, let’s look at the memory footprint.

Memory

When I saw that Cloud Sync requires Java to be installed, I knew that Cloud Drive might be something of a memory hog. Here is the RAM usage while each of the following clients are idle, taken just as a quick snapshot.

• EMC Syncplicity: 27.5 MB
• Mozy Stash: 17.3 MB
• Google Drive: 86.6 MB
• Dropbox: 44.5 MB
• Amazon Cloud Drive: 56.5 MB

Cloud Drive shows two related processes in Activity Monitor, so the number above totals the two. So, Cloud Drive does seem inefficient, but Amazon can rest easy next to Google Drive. (What are those self-important Mountain View PhD’s doing, anyway?!)

If Stash seems remarkably lean compared to the next two lowest, Syncplicity and Dropbox, I should note that Stash does not have some of the features that Syncplicity and Dropbox provide. For example, Stash lacks the right-click context menu on Mac OS (the platform from which I did my brief comparisons).

While fewer features may give Stash an advantage in memory utilization, no sync client runs leanest of all. My point is that the comparison is inherently skewed, and I’d like to have the menus. Nevertheless, Amazon Cloud Drive is feature poor compared to the others, so why does it have the 2nd highest utilization?

Over-the-Wire

The third realm of file sync efficiency is differential sync. When you update a file, does the software upload the whole file, or just the changes? Do other linked computers download the changes, or do they have to pull the whole updated file? I covered differential sync in Mozy Stash back in January. Dropbox put themselves on the map long ago with a video showcasing differential sync.

Amazon makes no mention of Differential Sync, so we must assume they don’t have it. Apparently, Google Drive does not offer differential sync either. But what does differential sync matter? The classic case for differential sync is the id3 tags in your music files. Say you add some album art, or correct a misspelled album title. That tiny change causes a sync engine to upload the whole big music file, and every other computer to download it. And since it was the album details, it’s not just one file. Only Dropbox and Stash handle this scenario with the extreme efficiency that makes a solid sync client. Since Amazon delivers the digital music they sell you by putting it directly into your cloud drive, differential sync seems especially important in the context of Cloud Drive. Maybe they’re working on it?

Conclusion

All in all, Amazon Cloud Sync is a good step for Amazon, but they have work to do on efficiency. To be sure, they’re not nearly so careless as Google Drive, but if this first release is any indicator, they need to knuckle down now if they want to avoid being yet another entrant that doesn’t really grok what differentiates a sync engine.

___

Note The observations and opinions I present above are my own. Mozy recently rejoined EMC, so EMC is now my employer. Mozy is working with the Syncplicity team, although I am not directly involved in that collaboration.

Working at Mozy

Although I have been blogging all too seldom in the past couple years, I want to share one of my responses for the “Best Companies to Work For in Utah” survey conducted annually by UtahBusiness.com. Why? Because I want to share with tech professionals in Utah what its like working at Mozy right now.

2010 in review

After this paragraph, the rest of the post is completely penned by WordPress.com. They give me seemingly high marks for my blog, while noting that I only posted 16 times in 2010. Remiss? Maybe. My top posts date back to early 2007 and before, when I was blogging actively for my role at Novell. But from the silver lining department, one of my top 5 referrers was bobjamieson.net, a paleo-geek’s blog. Perhaps a sign of things to come in 2011?

The stats helper monkeys at WordPress.com mulled over how this blog did in 2010, and here’s a high level summary of its overall blog health:

The Blog-Health-o-Meter™ reads Wow.

Crunchy numbers

About 3 million people visit the Taj Mahal every year. This blog was viewed about 40,000 times in 2010. If it were the Taj Mahal, it would take about 5 days for that many people to see it.

In 2010, there were 16 new posts, growing the total archive of this blog to 384 posts. There were 8 pictures uploaded, taking up a total of 198kb.

The busiest day of the year was January 5th with 206 views. The most popular post that day was Mac vs. PC: How Would Linux Fit?.

Where did they come from?

The top referring sites in 2010 were stumbleupon.com, roseindia.net, linuxcompatible.org, bobjamieson.net, and facebook.com.

Some visitors came searching, mostly for linux, wallpaper, mac, suse wallpaper, and opensuse wallpaper.

Attractions in 2010

These are the posts and pages that got the most views in 2010.

1

Mac vs. PC: How Would Linux Fit? March 2007
204 comments

2

Can Linux Desktops Live in an Active Directory World? September 2006
65 comments

3

OpenOffice.org and Excel VBA Macros July 2006
25 comments

4

SUSE Wallpaper #2 August 2006
1 comment

5

Show Me That New GNOME Main Menu June 2006
34 comments

Launching code.mozy.com

Since my start at Mozy in September, 2009, one of the internal programs in which I quickly took interest was Mozy Labs. Labs’ main champion was a former Google intern named JT Olds, who had witnessed directly the power of allowing engineers free time for innovation and wanted that for Mozy. After several months, Labs had spawned numerous projects, some of which are now on their way to becoming features for Mozy customers. But a few of the projects addressed lower-level needs in the Mozy service–such as helping Mozy handle massive storage (currently 50 petabytes) scale and data transfer demands. The Labs’ projects in this domain end up help us to serve our customers better, but are entirely the domain of deep-think developers. Nevertheless, the developers driving such projects want to share with others who would appreciate their innovations.

After several months of quiet preparations and effort,  we now have a way for those developers to do exactly that. Today, Mozy launched code.mozy.com, a site on which we can host free and open source software projects.

Related Resources

[updated 10/26]

• code.mozy.com
• Mozy blog: “Open Source and Mozy: The Debut of Mozy Code“
• Storagezilla: “Dive into Mozy Code“

Congrats to Polar Rose

Late last year, I met with Jan Erik Solem and Carl Silbersky to learn about Polar Rose‘s facial recognition technology. Their command of the tech, and stats compared to iPhoto and Picasa’s success rate, impressed me immensely. To the average MBA student, they likely seemed to be yet another tech startup with no real  business viability. If the competition are huge companies that make similar tech available at no cost, why would anyone pay?

But Jan Erik was very clear about Polar Rose: they were focused on R&D. They aimed to develop such a high mark of competency, that they would distinguish themselves through tech. The business would follow.

So when a former colleague at Mozy emailed me a link to “Apple Snatches Up Facial Recognition Software Firm,” it made me smile. They did it. They engineered their way to success. Congratulations, guys!

On API Replication and Web App Compatability

Commandeering Applications by Replicating APIs

I recently met with Jesse Stay to discuss some plans and ideas I have been working on at Mozy. As expected, Jesse’s insights and feedback were tremendously valuable. (Thanks, Jesse!)

One of the items Jesse directed my attention toward was the recent implementations of the Twitter API by three different sites: WordPress.com (the host of this blog), Tumblr (another popular blog hosting service), and identi.ca (a prominent Twitter alternative which has supported the Twitter API since 2008). By providing 100% API compatibility with Twitter, people can use any application built for Twitter with these other services (as long as the app allows you to modify the service URI/endpoint it uses). Because services survive or wither based on use, supporting a popular and relevant API can instantly yield a slew of applications to support users better. As a result of their work, WordPress.com has gained applications like the iPhone application Tweetie. Although many of the applications for Twitter currently do not allow you to modify the API endpoint (Tweetdeck, for example), most developers want more users, and API-level compatibility makes it short work for developers to support your service. I suspect that soon, nearly all applications created for Twitter will let you modify or add to the default API endpoint.

This got me to thinking about GroupWise, Novell’s strong alternative to Microsoft Exchange.

Novell: Align with Google for Developers

Novell GroupWise has a complete web API, and has for quite some time. Unfortunately, Novell seems to neglect developers these days–especially in their legacy products (as evidenced by GroupWise having only a SOAP API). Aside from their list of close partners, few developers use the GroupWise API.

Meanwhile, with Google’s enormous cachet with developers, numerous applications support Google’s collaboration-related APIs (such as Contacts). Perhaps even more importantly, developers can easily integrate Google’s API into new and existing apps through the numerous source code libraries available for Google’s various API’s (again, see Contacts, this time for the list of libs). Google’s Atom-based REST API’s are popular, well-understood, and widely used.

Consequently, Novell has an opportunity to serve their dwindling fan base better: replicate the Google API’s that are most relevant to GroupWise. Make it so that developers can easily adapt their apps to serve GroupWise, and so that end users can select from a wider assortment of 3rd party apps. This would give nothing over to Google that they don’t already have. It can only increase the relevance and extend the life of Novell’s venerable old product.

It also makes it possible for Novell to get on board with the biggest technology trend of the latter half of the last decade…

GroupWise in the Cloud

Since my time at Bungee Labs, I have been considering how Novell could offer GroupWise as a hosted SaaS offering as a way of acquiring aspiring new companies. As Salesforce has shown well, small is beautiful, and there is big business in the “No Software” proposition. This is where GroupWise competes (entirely ineffectively) with Gmail (“Google Mail,” actually) and its related collaboration components. It’s very easy to set up a Gmail account that uses your own company’s mail@domain. You can’t do that with GroupWise today.

But GroupWise does have something that Google lacks: a rich client. With all the apps that Google enjoys, none do a complete enough job of unifying all the typical collaboration features of an enterprise collaboration suite. With legacy collaboration being such an unsexy subject in the age of the social web, it seems unlikely that Google will re-solve some of these relatively mundane features. Finally, although there is a de facto rule about “everything through the web,” that does not mean “everything in the browser.” GroupWise has browser-based access, but it also has a full client that works online as well as offline, giving Novell a complete end-to-end advantage.

So put it all together–SaaS + Google API compatibility–and Novell has a viable technical path for GroupWise. All that remains is to figure out a  freemium model that can viably compete with Google. (This is where I clock out.)

Note I realize that it’s kind of cheap to sit back and throw out directives for how some other company should manage its API strategy while sparing any mention of the company over which I may have actual influence. Some of the above thinking applies to our planning at Mozy. However, we do not offer a public API (yet), so it’s a challenge to directly discuss this subject without getting mired in caveats and cautionary or indefinite statements. If you have any more than a casual interest in where Mozy is heading with our API, please contact me by commenting on this post.

Personal Cloud: A Microsoft Employee’s Take

An interesting and well articulated take on Personal Cloud from Vu Ha, a Microsoft engineer, states:

I believe that there is an excellent opportunity to build an open user-centered data platform that comprehensively addresses data silo and privacy issues, and thus catalyzes dramatic improvement in agent applications.

Me, too!

I recommend checking out his post.

Nat Friedman & the Personal Cloud: “Personal data warehouse”

Ximian co-founder and intrepid technologist with SUSE Linux, Nat Friedman recently blogged about a “Personal data warehouse,” stating:

What I want is a giant elastic bit bucket in the cloud, with a powerful search engine on top of it.

He goes on to describe several capabilities that he wants the search capabilities to have, essentially bringing together several disparate services available on the web today–such as face recognition (Polar Rose) and Optical Character Recognition  (OCR, the simplest form right now may be Evernote‘s)–in order to make his data imminently accessible and usable.

Nat describes several other aspects, all of which in my view comprise not a single service, but a data platform. This Personal Cloud concept really cannot be delivered well by a single service provider–you don’t want it to be. Once you have your personal data in the cloud, the next step is to have a selection of relevant applications to choose from for helping you to manage your Personal Cloud. That means APIs that allow developers to offer best-of-breed services, such as face recognition, as applications that you can use with your cloud-hosted personal data.

All of that reminds me that I really need to write up a post about the necessity for data owners (you and me as individuals) having ultimate control over who can access our data (and what data they can access).