I’m back. For real this time.
Quick up date: I work in NYC now building out lofty.com. Life has been chaos as usual.
Coming up:
Eight months! Apparently that is the magic number when it comes to the amount of time it takes for me to talk about work I’ve done and get motivated to polish it up a bit. That said, lets talk about django-simple-history! One of my bigger projects at work is a monstrous django project that I’ve built from the ground up that does a bunch of enterprise type nitty-gritty (yay CRUD). At time of inception, one of the big feature requests was to have tracking of _all_ changes to the database. This request included needs for the following:
I looked around and saw some basic implementations of different historical record attempts, including AuditTrail but had a requirement of needing models.OneToOneField & models.ForeignKey fields to work (store that there was a relation and not fail on lookups). I had read about something similar in Marty Alchin’s Pro Django book (highly recommend), so I dug that up and was sad to see that it also didn’t work with the relation fields. I reached out to Marty to see if he had a solution and it seemed that at the time he had not even thought about it too much past his initial work. I asked him if he’d be OK with me expanding upon his initial source and creating a project out of his work with changes to support the relation fields and he agreed. The one aspect that had to be left out for now is the tracking of the user who made the change. This _can_ be done. I do it with an extra field added to the history model. That said, hopefully soon I can release that code with a patch to django-simple-history so you can easily do this. For now please contact me for more details about tracking the current user making changes. That said, everything else was free game so I threw it all together and put it on bitbucket.
Thus django-simple-history was born.
I kinda threw the code up after i got it working and totally forgot about it. I’m sad to admit that I didn’t do it due diligence, even if I never planned to make it a package. That was until someone actually forked my code! *SHOCK* They had added a setup.py, so I pulled it in and expanded on it (it was pretty simple at first). Now there is a fully installable app with example code in the README and an actual package uploaded to make django-simple-history 1.0!
Here’s the basic rundown on how you use it using the django tutorials Poll and Choice models (used django 1.2.3 for these tests):
from django.db import models
# import the HistoricalRecords model
from simple_history.models import HistoricalRecords
class Poll(models.Model):
question = models.CharField(max_length = 200)
pub_date = models.DateTimeField('date published')
# create an instance of HistoricalRecords on any model you want to track
history = HistoricalRecords()
class Choice(models.Model):
poll = models.ForeignKey(Poll)
choice = models.CharField(max_length=200)
votes = models.IntegerField()
history = HistoricalRecords()
That’s it! Once you’ve done this, when you $ ./manage.py syncdb you will get the following:
BEGIN;
CREATE TABLE "test_historicalpoll" (
"id" integer NOT NULL,
"question" varchar(200) NOT NULL,
"pub_date" datetime NOT NULL,
"history_id" integer NOT NULL PRIMARY KEY,
"history_date" datetime NOT NULL,
"history_type" varchar(1) NOT NULL
)
;
CREATE TABLE "test_poll" (
"id" integer NOT NULL PRIMARY KEY,
"question" varchar(200) NOT NULL,
"pub_date" datetime NOT NULL
)
;
CREATE TABLE "test_historicalchoice" (
"id" integer NOT NULL,
"poll_id" integer,
"choice" varchar(200) NOT NULL,
"votes" integer NOT NULL,
"history_id" integer NOT NULL PRIMARY KEY,
"history_date" datetime NOT NULL,
"history_type" varchar(1) NOT NULL
)
;
CREATE TABLE "test_choice" (
"id" integer NOT NULL PRIMARY KEY,
"poll_id" integer NOT NULL REFERENCES "test_poll" ("id"),
"choice" varchar(200) NOT NULL,
"votes" integer NOT NULL
)
;
CREATE INDEX "test_historicalpoll_4a5fc416" ON "test_historicalpoll" ("id");
CREATE INDEX "test_historicalchoice_4a5fc416" ON "test_historicalchoice" ("id");
CREATE INDEX "test_historicalchoice_763e883" ON "test_historicalchoice" ("poll_id");
CREATE INDEX "test_choice_763e883" ON "test_choice" ("poll_id");
COMMIT;
* test was my app name for use in testing… poor choice.
The two historical db’s were created with the extra fields needed to track them historically:
Now to use, fire up the django shell $ ./manage.py syncdb and try this!
In [2]: from poll.models import Poll, Choice In [3]: Poll.objects.all() Out[3]: [] In [4]: import datetime In [5]: p = Poll(question="what's up?", pub_date=datetime.datetime.now()) In [6]: p.save() In [7]: p Out[7]: <Poll: Poll object> In [9]: p.history.all() Out[9]: [<HistoricalPoll: Poll object as of 2010-10-25 18:03:29.855689>] In [10]: p.pub_date = datetime.datetime(2007,4,1,0,0) In [11]: p.save() In [13]: p.history.all() Out[13]: [<HistoricalPoll: Poll object as of 2010-10-25 18:04:13.814128>, <HistoricalPoll: Poll object as of 2010-10-25 18:03:29.855689>] In [14]: p.choice_set.create(choice='Not Much', votes=0) Out[14]: <Choice: Choice object> In [15]: p.choice_set.create(choice='The sky', votes=0) Out[15]: <Choice: Choice object> In [16]: c = p.choice_set.create(choice='Just hacking again', votes=0) In [17]: c.poll Out[17]: <Poll: Poll object> In [19]: c.history.all() Out[19]: [<HistoricalChoice: Choice object as of 2010-10-25 18:05:30.160595>] In [20]: Choice.history Out[20]: <simple_history.manager.HistoryManager object at 0x1cc4290> In [21]: Choice.history.all() Out[21]: [<HistoricalChoice: Choice object as of 2010-10-25 18:05:30.160595>, <HistoricalChoice: Choice object as of 2010-10-25 18:05:12.183340>, <HistoricalChoice: Choice object as of 2010-10-25 18:04:59.047351>]
The highlighted lines show how to access the history of each object:
Please leave a comment if you have any questions about this, or drop a bug report on bitbucket. I’m open to suggestions and/or more help to fill out some of the other relational field types (ManyToMany).
tl;dr – Use django-simple-history, it’s epic.
I’ve been using Jesper Noher’s django-piston project for some projects at work and home and I must say, I _really_ love it. Sadly, I don’t think the project gets as much credit as it should. If you’re looking to add REST to django, you really cant go wrong with piston.
That said, as I started to use it more and more, I hit some roadblocks that I had to come up with solutions to.
TLDR; Checkout my additions to django-piston here http://hg.qr7.com/django-piston/.
The @validate decorator in piston.utils worked great for creating new objects (POST), but didn’t really work at all when trying to update models (PUT). I extended this function to accept a model reference and define which fields are passed in through the url for existing object lookup. Yea, thats not clear, lets throw together an example (figure out your own imports). ***I haven’t tested this at all… let me know if i screwed up somewhere.
The model (models.py:
class MyModel(models.Model):
testfield = models.CharField(max_length=50, unique=True)
And a form for validation (forms.py):
class MyModelForm(forms.ModelForm):
class Meta:
model = MyModel
Cant work without urls’s (urls.py):
basic_auth = HttpBasicAuthentication(realm='My Realm')
basic_mymodel = Resource(handler=MyModelHandler, authentication=basic_auth)
urlpatterns = patterns('',
url(r'mymodel/$', basic_mymodel),
url(r'mymodel/(?P<testfield>[^/]+)/$', basic_mymodel),
)
Finally, the handler (handlers.py):
class MyModelHandler(BaseHandler):
"""
Authenticated entrypoint for MyModel entries.
"""
model = MyModel
anonymous = 'AnonymousMyModelHandler'
fields = ('testfield',)
allowed_methods = ('GET', 'POST', 'PUT')
def read(self, request, *args, **kwargs):
base = self.model.objects.all()
if args != () or kwargs != {}:
try:
return base.get(*args, **kwargs)
except self.model.DoesNotExist:
return rc.NOT_FOUND
except self.model.MultipleObjectsReturned:
return rc.BAD_REQUEST
else:
return base
@no_params()
@validate(MyModelForm, 'POST')
def create(self, request, *args, **kwargs):
if not hasattr(request, "data"):
request.data = request.POST
attrs = self.flatten_dict(request.data)
try:
mymodel = self.model(testfield=attrs['testfield'])
except:
return rc.BAD_REQUEST
else:
mymodel.save()
return mymodel
@essential(['testfield'])
@validate(MyModelForm, 'PUT', ['testfield'], MyModel)
def update(self, request, *args, **kwargs):
if not hasattr(request, "data"):
request.data = request.POST
try:
instance = self.queryset(request).get(fqdn=kwargs['mymodel'])
except self.model.DoesNotExist:
return rc.NOT_FOUND
except self.model.MultipleObjectsReturned:
return rc.BAD_REQUEST
except:
return rc.BAD_REQUEST
attrs = self.flatten_dict(request.data)
for k,v in attrs.iteritems():
setattr(instance, k, v)
instance.save()
return instance
As you can see on the highlighted lines, I’ve modified @validate to accept more params, and added the @no_params and @essential decorators. The three break down like this:
@validate now takes in an optional list of lookup fields, these should map to your urls.py as the inputs. In the example i have @no_params is used to ensure that no parameters have been passed in, in this example, I want only POST’s to mymodel/ to be accepted for object creation. Without this, sometimes @validate will return weird errors that you I didn’t want the user to see. Doing the check in the actual create function wouldn’t work because @validate wraps it first and takes precedence.@essential is used to ensure that the passed in list of parameters have been passed in to the request. In this case, testfield is essential for the existing object lookup (ok ok this is a bad example, I’m modifying a unique identifier, but thanks to django’s form validation, the new testfield should be checked for uniqueness as well!). Once we’ve ensured that testfield exists as a param, we can then validate against the form. I guess this is somewhat of a bad example, as the form will bitch that testfield is needed, but there are cases where the form bitches without any helpful information. I like having the option of presenting bad requests to users before the form displays errors. This serves that purpose very well.All of this work was pretty straight forward. I went ahead an forked django-piston and made my changes to utils.py. You can find it all here: http://hg.qr7.com/django-piston/. I’ve issued a pull request to trunk and hopefully some/all of these will get added with any hope. If not, there here if you find yourself bumping up against the same problems. I’d be interested in anyones thoughts/contributions on all of this. Please drop me an email, comment, or PM me on bitbucket/twitter/wherever.
Hope this helps!
