Team:Berkeley/Clotho

From 2010.igem.org

Latest revision as of 02:18, 28 October 2010

Clotho

The UC Berkeley iGEM Team of 2008 started the Clotho software project to provide a suite of software tools for synthetic biology. Since then, Clotho has evolved into a platform on which new software tools can be developed. Clotho as a platform factors out the common functionality that many software tools would provide, thus allowing new tools to be developed more quickly and making it easy to integrate new tools with existing tools. More importantly, anyone can now write a tool that will work seamlessly with other Clotho tools. In this regard, you can think of Clotho like the an iPhone. Anyone can write an App, and the platform facilitates its creation, distribution, and operation. The Clotho project has also expanded beyond the walls of UC Berkeley to include collaborators in the BIOFab, JBEI, Boston University, University of Washington, Autodesk, and the DIY community. Perhaps you’ll join the movement as well!

This summer, the 2010 Berkeley iGEM Team is proud to release Clotho 2.0 which completes Clotho’s transition to a community resource for open tool development based on a standardized API. Though the overall effort included the work of many folks outside the iGEM team, our iGEM group was responsible for developing the new internal architecture of Clotho and implementing its first set of Apps. Clotho 2.0 now features the robust Netbeans plugin platform App platform, a reworked framework for automating workflows framework for automating workflows, as well as a shift from dynamic data models to a static Clotho data model Clotho data model. Read more about these new features here on our wiki, or visit our new public [http://www.clothocad.org/ website] for more information and to download our wonderful tool!

App Platform

Clotho provides an App framework for the development of new tools. As illustrated by the picture below, Clotho Apps are built on top of the Clotho API, and are able to use functionality provided by this API. Clotho in turn uses functionality from other libraries below it. The App framework is built on the Netbeans Platform (([http://netbeans.org netbeans.org]), which manages the App environment. This includes checking that all the dependencies of an App are satisfied before it is launched, managing all the files that may be necessary for a App to function correctly, as well as actually loading the executable code for each App into Clotho at runtime. The Hibernate library provides a structured interface to external databases and is discussed in more detail on the Data Model page.

High level schematic of the Clotho architecture. The Clotho API servers as a bridge between the more general frameworks and the Apps. This allows Clotho to provide an interface with domain specific functionality that tools made for synthetic biology can leverage.

Clotho classifies Apps into seven distinct categories: ClothoAlgorithm, ClothoConnection, ClothoFormat, ClothoGrammar, ClothoTool, ClothoViewer, and ClothoWidget. These categories are based on the different types of functionality a App can provide. For example, a ClothoConnection provides connections to a database, while a ClothoTool is a tool built on the Clotho platform. Apps are discovered and loaded dynamically when Clotho starts. For more information about how to write a App, what functionality the Clotho API provides, and to download Apps for your own use, visit the [http://clothocad.org clothocad.org].

Automated Workflows

Automating synthetic biology workflows has always been a goal of the Clotho project. As devices become increasingly complex and utilize greater numbers of parts, it makes more and more sense to automate many of the steps in the construction of these devices. Last summer, the Berkeley iGEM team sought to build an automation framework for synthetic biology on top of Kepler, a tool for automating scientific workflows developed by the CHESS group at Berkeley ([http://chess.eecs.berkeley.edu/ chess.eecs.berkeley.edu]). Kepler is a powerful tool, but its large size and complexity makes it hard to integrate with Clotho. Additionally, the use of Kepler had been restricted to its use in the context of Automated Assembly as a plugin to Clotho's core functionality. Because we believe that this type of program would be useful beyond one specific application, we sought a new strategy for integrating the core functionality of Kepler into the core functionality of Clotho. So we decided to recode a simplified automation framework using the ideas from the Kepler project.

A single Actor object in Clotho. The green boxes represent InputPorts, which describe the inputs to the Actor, while the red boxes represent OutputPorts. Each InputPort or OutputPort specifies the type of data it receives or outputs, respectively. This allows Actors to be chained together to form workflows

What we came up with is illustrated above. A unit of automation is called an Actor, which executes some algorithm. Each Actor defines a set of InputPorts and OutputPorts, and each Input/Output port only works with one type of data. Thus, the set of ports of an Actor specifies how it can interact with other Actors. Actors allow developers to reuse code from other developers, as well as allowing users to chain together Actors written by different developers into new workflows, customized for their own needs.

Multiple actors can be chained together to form a workflow. Clotho is able to check that all InputPorts not connected to an OutputPort are fed some initial data, after which the Actors are exectued in order.

A workflow is built by connecting the inputs and outputs of multiple Actors. After a workflow has been constructed and fed initial data, Clotho will execute the Actors in the workflow according to the partial ordering defined by the workflow (which is a directed acyclic graph). Users can construct workflows using the scripting interface provided by Clotho. In addition, a graphical interface to build workflows is currently under development by the Clotho team. These tools will make it easy for users to share their automated workflows with the community.

Data Model

Potentially the most fundamental change this summer was a shift from dynamic data models to a static Clotho data model. In previous years, Clotho tried to remain data model agnostic by allowing users to define their own data model. However, this introduces major problems for developers trying to build tools for Clotho. If one tool is built assuming one data model, then it is unlikely to be compatible with another tool built assuming a different data model. Even worse, if a user changes the data model after installing a tool, the tool could be rendered nonfunctional.

Because we could see no good solution to the problems generated by a dynamic data model, we adopted a new architecture that presents two layers of data exchange. Tools directly receive and send data using a standardized datamodel to Clotho's Core, and then the Core exchanges data through ClothoConnection Apps to databases using whatever schema is encoded in the database. In this way, we get the best of both worlds. Tools all work with the same standard data model when working atop the Clotho API, but databases based on a different data model can still talk to Clotho via customized ClothoConnection Apps. This architecture guarantees that different tools written by different people nevertheless interpret a "Part" the same way. This makes the exchange of data within Clotho across tools not only possible but a fluid and natural to the user. The standardized API also allows Clotho to validate data before it is saved to a database and shared with other users, which is especially important in the light of biosafety. More information is available on our Human Practices page about how Clotho makes doing synthetic biology safer.