Live Mesh as a Platform

I've been undercover for two years. After helping to get XNA off the ground, I took an opportunity to learn some new things from some very good people. The pain of working on version 1s fades after a few years, so I decided to try another one. The pain is again fresh in my mind. But that’s not what this post is about – it’s about the excitement of launching version 1.

This project requires deep domain experts on distributed systems, scale data centers, web, Windows, Mac, mobile, networking, app protocols, file systems, databases, synchronization, peer to peer, security and more. We not only had to find these people and convince them that it would be cool to pull heavy Gs on a version 1 “for a year”, we had to establish a common vocabulary and then all agree on some things. One of the very cool things about Live Mesh is that is has a broad scope, but is at the same time respectful of the specialization needed to succeed on the web, Windows, Mac and mobile. We couldn’t have achieved this without a team with deep but different experience, knowledge, practices, jargon and opinions – and that still managed to cooperate even when it got hard.

We introduced Live Mesh at the Web 2.0 conference in San Francisco this evening. We are shipping working code to a community size that we think we can support while still working on the product. I look forward to the feedback.

We operate at such a scale at Microsoft that we have developed an instinct to specialize and focus. One of the typical early conversations in a new project is the target audience: consumers or enterprise? Large or small organizations? Web, Windows, Mac or mobile? What mobile targets? We rejected this early and focused us on adding value to individuals: people who may work in enterprises and belong to multiple organizations, but who also make choices as consumers, and use multiple technologies (and who are probably frustrated with the productivity barriers that exist as a side effect of the seams the industry imposes on them). We’ve thought a lot about adding value to enterprises too, primarily by empowering individuals while keeping in mind the confidentiality expected by organizations.

Our goal was to build a platform that spanned individuals, organizations and technologies. But we didn’t want to fall into the trap of assuming that we could get it right without using and proving it ourselves. And we also believed that the easiest way to express the fundamental potential of a platform is with an experience that delights and generates even more ideas.

To that end, we built both a platform and a set of basic horizontal experiences on multiple endpoints in parallel. As someone primarily concerned with the platform, the opportunity to “see it in action” as we build it has been very valuable and made the process more fun. I like them because they blend the web, Windows and other computing endpoints in a way that preserves the “it just works” feel of the web with seamless integration into my common workflows.  The coolest thing about Live Mesh is how it smashes the abrupt mental switch that I have to make today as I move between being “on the web” and “in an application”.

This post is about the Live Mesh platform. We are not yet ready to release the Live Mesh platform, but I would like to share some of the core metaphors and building blocks we’ve been working on.

The focus for Live Mesh version 1 is to establish a Software + Services platform and ecosystem that enables unique new customer value, is unexpected by our competitors and will support broad innovation across multiple future releases. The core philosophy is to make it easy to manage information in a world where people have multiple computing experiences (i.e. PCs and applications, web sites, phones, video games, music and video devices) that they use in the context of different communities (i.e. myself, family, work, organizations).

My favorite example of a Software + Services application is Live Messenger (iTunes is another good example). The customer experience combines the best of local applications: always running, integrated with the shell and other applications, able to do background work and access local peripherals with the easy access to information that the web provides. Mesh is a Software + Services platform.

At the core of Mesh is concept of a customer’s mesh, or collection of devices, applications and data that an individual owns or regularly uses. The Mesh Account Service persists the relationship among these resources and authorizes access to them. The mesh is the foundation for a model where customers will ultimately license applications to their mesh, as opposed to an instantiation of Windows, Mac or a mobile account or a web site.  Such applications will be seamlessly installed and run from their mesh and application settings persisted across their mesh. The device ring inside of the Live Desktop is a simple visualization of the mesh, and provides a view of all devices and current device availability. The Live Mesh platform provides the ability for applications to connect to any other device, regardless of network topology (network transparency), within the mesh. This infrastructure enables the Live Mesh Remote Desktop experience today.

A key design goal of the Live Mesh data synchronization platform is to allow customers to retain the ownership of their data that is implicit with local storage while improving on the anywhere access appeal of the web. The evolution of the web as a combined experience and storage platform is increasingly forcing customers to choose between the advantages of local storage (privacy, price, performance and applications) and the browser’s implicit promise of data durability, anywhere access and in many cases, easy sharing. A side effect of the competition to store customer data in the cloud and display it in a web browser is the fragmentation of that data and subsequent loss of ownership. Individual sites like Spaces, Flickr and Facebook make sharing easy, provided the people you are sharing with also use the same site. It is in fact very difficult to share across sites and equally difficult to work on the same data across the PC, mobile and web tiers.

The fundamental unit of synchronization in Live Mesh is a mesh object. Access to mesh objects is via the common web metaphor of a feed. These building blocks provide low barriers to adoption in addition to enabling appropriate customer experience metaphors in different technology, organization and business domains. As an example, one instantiation of a mesh object is as a local (shared, aka Live) folder on a PC. This same mesh object might be instantiated as a slideshow on a web site, and as preview and upload UX on a mobile device with a built-in camera. A Live Folder is but one specialization of a mesh object. A mesh object could also represent a range of cells in Excel or a To Do list that can be accessed from anywhere. You’ll see more on this in an upcoming Channel 9 interview with our storage and synch team.

Live Mesh has been designed with the goal of enabling explicit control over the location and custody of data even in a sharing environment. Mesh objects (i.e. files, lists, etc. and associated information) are physically stored in a Live Mesh Storage Server and exposed as authenticated feeds. The Live Mesh cloud implementation of a storage server is intended to be a peer to large number of Mesh Enterprise Storage Servers that can be offered in subsequent releases. This storage and feed architecture supports a completely enterprise scoped sharing across all tiers: PC, intranet browser, and mobile in addition to home - home and home - work scenarios.

Live Mesh provides the building blocks to support the notion of groups, or communities (member lists) of people associated with a mesh object. These lists are managed via the simple email-web confirmation exchange that is popular on the web. Member lists provide both a simple permission model and a natural scoping for the collaboration features of Mesh. The basic group mechanism can be trivially extended to add entities such as organizations to participate (in an assigned “role”).  Similarly, you might invite a SpellChecker bot into a group.

Every layer of the mesh is built on top of APIs in the form of feeds. These layers enable:

The ability to add / remove devices (and someday applications) to a mesh and manage groups (by initiating invitations).

The ability to open a mutually authenticated raw communications channel, to any device in a group, regardless of current location or network topology. This channel always works, by way of cloud relay if necessary, but will automatically and transparently take the cheapest and fastest possible network path.

The ability to subscribe (“tell me when it has changed”) to resources and publish (“I changed it”) notifications against resources. The pub/sub infrastructure eliminates polling the cloud to become aware of changes. The sync engine itself uses pub/sub for sync change triggering.

An extensible “awareness” infrastructure which is scoped to the group and the information being shared. The first Live Mesh experiences provide a very simple notion of awareness – who is “using” a Live Folder now, but the infrastructure is designed to support much richer notions. The programmer sees a “durable for a login session” extensible XML document (unique per mesh object/group) that can be queried/updated at a very high rate and scale. Like everything in Live Mesh that needs to be queried, the programmer uses the pub/sub engine to signal interest in changes, and waits for notification of such changes.

A durable news event stream. Unlike awareness, which is level triggered against a schema, news is message based. The goal of news and awareness are to create a very real time intimacy around the collaboration experience.

A FeedSync based sync infrastructure that enables any device that is capable of web protocols to participate in bidirectional sync, again in the context of (fully authenticated) groups. One of the “devices” that can participate in this experience is a 3rd party web site. While the version 1 experience is primarily around syncing folders of files, FeedSync and the Live Mesh infrastructure is far more flexible. It could sync lists, tables, photo galleries, playlists, calendars, etc.

A scale blob store that can be referenced (as an enclosure) in the sync feed.

Because this is all built on feeds, all of this functionality is accessible from every endpoint that can run a browser (i.e. darn near everything). The Live Desktop, Windows Live Folders, Mac and mobile experiences are the first examples of the potential of this platform.

It’s just the beginning.

Mike Zintel

Product Unit Manager, Mesh and Storage Platform

Technorati Tags: LiveMesh