The base QuantumBuster installation is already build on the basic products but can also be extended by third party products at any time. Problems involved with product management are:
- Installation
- Dependencies
- Upgrades / Migration
- De-Installation
Moreover in QuantumBuster we have a flexible storage machinery and thus how to implement the storage layer is an additional problem.
A product is simply a Python package which registers with QuantumBuster. It is doing that by registering a utility which returns more information about the product if asked for it. These are things like:
- product name
- product description
- product version
- dependencies
Please note that a product usually comes without the storage part, this will be a separate package (or multiple for multiple types of storages). Thus the main product package mainly defines the data model and the product logic while the storage package will serialize the data model to whatever storage it wants to.
As the storage is flexible care must be taken that you don’t use e.g. foreign keys across products in SQL based storages because the product you refer to might not be stored either in the same SQL database or in any SQL database at all (e.g. ZODB, CouchDB, Amazon SimpleDB etc.).
The Product Information Utility is used to describe a product and it’s interface looks like follows:
class IProductInformation:
id = String('''unique string identifying the product suitable for named interfaces''')
name = String('''the human readable name of the product''')
description = String('''a short description of that the product does''')
version = Float('''the version of the product''')
requires = [] # list of python package name string which this product requires to work with. This MUST NOT include the storage package as these are meant to be replaceable.
This is very similar to the information provided for an egg and usually can be reused.
Each utility is registered with the component registry by it’s unique id and the interface (thus being retrievable as a utility with a named interface)
Note
Probably more information is needed such as provided content types so that the product contens can be shown in some menu to add content of that product. This will get added later when we describe content types.
If you install the product package the product is not actually available to QuantumBuster. You first need to install it within QuantumBuster. Two things are needed for this:
- You need to include the ZCML of the product and the preferred storage for it in your main ZCML file.
- You need to visit the installation screen to either install or upgrade the product
Step 1 is needed so that QuantumBuster actually has access to the product contents, esp. the Product Information Utility. Step 2 is needed so that QuantumBuster can add the product and esp. it’s version. In case you upgrade the product to a new version QuantumBuster can thus check which products need to run the upgrade routines.
Note
It’s still unclear yet on how you select the storage. Maybe storages need another utility describing the storage and the product it’s compatible with. It might have different named utilities for initializing or upgrading it’s storage. It also needs to have access to some global storage connector, e.g. an SQL connect string or a utility providing a database connection (maybe with different permissions). So probably we also need named utilities as e.g. ‘ IStorage with different names like storm, sqlalchemy, zodb and so on. The question is though if the interface can be the same on all of them. Maybe it needs to be sub interfaces instead of named ones. One storage provider package might then provide the database connection with a sub interface, get configuration from the ini file and can be required from the storage packages.
On the installation view of QuantumBuster all products (installed and not installed ones) will be listed along with the actual choice of storage package. Also displayed will be each version and if an upgrade is required or not. If so it can be initiated via this screen. Moreover uninstalled packages can be installed and installed ones can be deinstalled (with a question of the storage should also perform a deinstall).
Note
If content types are registered then those should be ignored for uninstalled products which still have content types in the database. Another problem might arise if a product provides e.g. a Folder type which has instances containing other objects. Deleting this is problematic then which might require a check_uninstall() method in each product which in turn means that some means need to be there to retrieve a list of objects in the database (but the nodes might be helpful here as they can be quickly queried for content types. It’s matter of interface design though).
The Product Storage provides the actual object which the Product package is working on with adapters. This a Storm based storage object might look like this:
class Homepage(object):
implements(IHomepage)
__storm_table__="homepage"
content_id = Int(primary=True)
title = Unicode()
description = Unicode()
content = Unicode()
def __init__(self, node = None, title=u"", description=u"", content=u"", **kw):
"""create a new homepage"""
self.title = title
self.description = description
self.content = content
self.node = node
Apparently it must store those fields define in the homepage definition. Moreover only the __init__ method should be implemented.
The Product package itself will then used adapters based on IHomepage to attach functionality to it.
Note
We might have a problem here with automatically generated content types such as TTW. Here we cannot used utilities or at least we shouldn’t add too much magic. So this needs more thinking to make sense. We also should take care that not too many versions of some data definition are around (e.g. form and interface definition).
As layed out above a product basically consists of two packages: The actual product and the storage module. Both need to implement some utilities to function.
Here is a step-by-step description of how to create a product.
This is simply a python package so you can basically create one with e.g. paster create -t basic_package or use the basic_namespace` which can be found in the ZopeSkel package. You can also use the paster template provided by the qb.core` package. This provides stubs for the necessary utilities.
The first thing you should do is create a utility which provides information about your product. This can look like follows inside a productinfo.py file:
from zope.interface import implements
from qb.core.interfaces import IProductInformation
class ExampleProductInformation(object):
"""a utility describing our example product"""
implements(IProductInformation)
id = u"exampleproduct" # TODO: Could this simply be the package name?
name = u"Example Product"
description = u"longer description of the example product, of what it does etc."
version = 1.0
requires = [] # qb.core etc. are automatically required
content_types = [u'ExampleType']
Then this needs some registration inside a configure.zcml:
<configure
<utility
factory = ".productinfo.ExampleProductInformation"
name = "exampleproduct"
/>
</configure>
In your main configure.zcml you’d then need to include the product’s configure.zcml:
<include package=”exampleproduct” />
Now we need a further utility which describes the content type. This could be places in content_type.py:
from zope.interface import implements
from qb.core.interfaces import IContentType
from qb.core.content_type import BaseContentType
from form import example_type_form
from model import ExampleTypeObject
class ExampleContentType(BaseContentType):
"""an example content type"""
implements(IContentType)
content_type = u"exampleproduct.ExampleType"
title = u"Example Type"
description = u"an example content type"
filter_subtypes = True
allowed_subtypes = [u'exampleproduct.ExampleType']
form = example_type_form # form being used for adding and editing objects of this type
model = ExampleTypeObject # object which implements this content type
Next up we need to define the form in form.py:
from tw.forms import *
from tw.tinymce import TinyMCE
from tw.forms.validators import *
from tw.api import WidgetsList
from formencode import FancyValidator, Invalid, All
from qb.twforms.baseform import *
class fields(WidgetsList):
title = TextField(
description="Please enter the title of this object",
title="Title",
size=80,
default="",
validator=String(not_empty=True))
description = TextArea(
description="Please enter a short description of this object",
title="Description",
cols=80, rows=3,
default="",
validator=String())
content = TinyMCE(
description="The content of this object",
title="Content",
cols=80, rows=30,
default="",
validator=String())
class ExampleTypeForm(BaseForm):
fields = standard_fields + fields
example_type_form = ExampleTypeForm(id="exampleproduct_exampletype")
Now we just need to configure this content type in configure.zcml:
<utility
factory=".content_type.ExampleContentType"
name="exampleproduct.ExampleType"
/>
Now we need a component which provides the actual data object for storing the content type’s data. This is done in a separate package with the name usually of the form exampleproduct.sql. So for our ExampleType type we now need to define a storage. We show this by providing a Storm based storage and a ZODB based storage (then under the name of exampleproduct.zodb).
First of all we need to define the data object which we do in model.py:
from zope.interface import implements
from Storm.locals import *
from qb.core.interfaces import IStorageObject
class ExampleTypeObject(Storm):
"""a storage object for instances of example type"""
implements(IStorageObject)
object_id = Int(primary = True)
title = String()
description = String()
content = String()
def __init__(self, node = None, title=u"", description=u"", content=u"", **kw):
"""create a new example type object"""
self.title = title
self.description = description
self.content = content
self.node = node
This simply defines an object with the three attributes title, description and content. The lines above define those attributes for Storm so they get serialized. Those mirror the attributes we have define in our form. object_id is not part of that list but is used as primary key. Every content type object in QuantumBuster needs to have this attribute because this is used to link content objects to nodes.
__storm_table__ now simply defines the table to be used by Storm for serializing it. Apparently this table needs to contain the attributes defined.
Now that we have the actual data container object we also need a component for creating new objects, retrieving existing ones and deleting old ones. These methods are defined in the IStorage interface. Here is our implementation in storage.py:
from zope.interface import implements
from zope.component import getUtility
from qb.core.interfaces import IStorage
from qb.storage.storm import IDB
from model import ExampleTypeObject
class ExampleTypeStormStorage(object):
"""implements a storage for example type"""
implements(IStorage)
def new(self, node, **values):
"""create a new example type"""
obj = ExampleTypeObject(node, **values)
db = getUtility(IDB)
store = db.get_store()
store.add(obj)
store.flush() # make sure it has an id
return obj
def get(self, node, id):
"""return the content object with the given id for the given node"""
db = getUtility(IDB)
store = db.get_store()
obj = store.get(ExampleTypeObject, id)
# attach the object to the node it's linked to.
obj.node = node
return obj
def del(self, object):
"""delete the given object"""
TODO: TBD
TODO. Description
Now we can register these components so that QuantumBuster knows about the storage and content types could be used. Again this happens in configure.zcml:
<utility factory=".storage.ExampleTypeStormStorage"
name="exampleproduct.ExampleType"
/>
Again we use a named utility for defining the storage for the content type. The naming convention is again <productpath>.<content type name>. Whenever you need to obtain an object you can use it as follows:
from zope.component import getUtility
from qb.core.interfaces import IStorage
storage = getUtility(IStorage, name=u"exampleproduct.ExampleType")
obj = storage.get(None, 13) # retrieve object with object id 13 and attach None as node
obj = storage.new(None) # create a new object and attach None as node
storage.del(obj) # delete the object (might crash here because it's not committed yet)
This is also used internally by QuantumBuster and usually you don’t need to use these methods. You will more likely create a new node and object in one step.
And here is the actual implementation for the ZODB, again first model.py:
from ZODB.persistent import Persistent
from qb.core.interfaces import IStorageObject
from zope.interface import implements
class ExampleTypeObject(Persistent):
"""ZODB implementation"""
implements(IStorageObject)
def __init__(self, node = None, title=u"", description=u"", content=u"", **kw):
"""create a new example type object"""
self.title = title
self.description = description
self.content = content
self._v_node = node
@property
def node(self):
"""return the node this object is being attached to. We have to use a volatile variable here because
we don't want the node to end up in the ZODB, too"""
return _v_node
TODO: continue with storage.py
..note :: We need to think again about IStorageObject because we might want to have one storage object implementation for multiple content types (e.g. for TTW work) while we also might want to attach different adapters to different content types. Here we only have one interface defined though.