OHHHHKAY

I’m back. For real this time.

Quick up date: I work in NYC now building out lofty.com. Life has been chaos as usual.

Coming up:

  • updates to django-simple-history
  • will include adding the current user to a change
  • news about my new job @ lofty.com
  • have a lot of thoughts do dump about startups and python’s place in that world
  • a few tools to release (some are p. nifty)
  • new side projects starting up
Continue reading » · Written on: 05-31-11 · No Comments »

django-simple-history FTW!

Eight months! Apparently that is the magic number when it comes to the amount of time it takes for me to talk about work I’ve done and get motivated to polish it up a bit. That said, lets talk about django-simple-history! One of my bigger projects at work is a monstrous django project that I’ve built from the ground up that does a bunch of enterprise type nitty-gritty (yay CRUD). At time of inception, one of the big feature requests was to have tracking of _all_ changes to the database. This request included needs for the following:

  • change type (create, update, delete)
  • change time
  • change user

I looked around and saw some basic implementations of different historical record attempts, including AuditTrail but had a requirement of needing models.OneToOneField & models.ForeignKey fields to work (store that there was a relation and not fail on lookups). I had read about something similar in Marty Alchin’s Pro Django book (highly recommend), so I dug that up and was sad to see that it also didn’t work with the relation fields. I reached out to Marty to see if he had a solution and it seemed that at the time he had not even thought about it too much past his initial work. I asked him if he’d be OK with me expanding upon his initial source and creating a project out of his work with changes to support the relation fields and he agreed. The one aspect that had to be left out for now is the tracking of the user who made the change. This _can_ be done. I do it with an extra field added to the history model. That said, hopefully soon I can release that code with a patch to django-simple-history so you can easily do this. For now please contact me for more details about tracking the current user making changes. That said, everything else was free game so I threw it all together and put it on bitbucket.

Thus django-simple-history was born.

I kinda threw the code up after i got it working and totally forgot about it. I’m sad to admit that I didn’t do it due diligence, even if I never planned to make it a package. That was until someone actually forked my code! *SHOCK* They had added a setup.py, so I pulled it in and expanded on it (it was pretty simple at first). Now there is a fully installable app with example code in the README and an actual package uploaded to make django-simple-history 1.0!

Here’s the basic rundown on how you use it using the django tutorials Poll and Choice models (used django 1.2.3 for these tests):

from django.db import models
# import the HistoricalRecords model
from simple_history.models import HistoricalRecords

class Poll(models.Model):
    question = models.CharField(max_length = 200)
    pub_date = models.DateTimeField('date published')

    # create an instance of HistoricalRecords on any model you want to track
    history = HistoricalRecords()

class Choice(models.Model):
    poll = models.ForeignKey(Poll)
    choice = models.CharField(max_length=200)
    votes = models.IntegerField()

    history = HistoricalRecords()

That’s it! Once you’ve done this, when you $ ./manage.py syncdb you will get the following:

BEGIN;
CREATE TABLE "test_historicalpoll" (
    "id" integer NOT NULL,
    "question" varchar(200) NOT NULL,
    "pub_date" datetime NOT NULL,
    "history_id" integer NOT NULL PRIMARY KEY,
    "history_date" datetime NOT NULL,
    "history_type" varchar(1) NOT NULL
)
;
CREATE TABLE "test_poll" (
    "id" integer NOT NULL PRIMARY KEY,
    "question" varchar(200) NOT NULL,
    "pub_date" datetime NOT NULL
)
;
CREATE TABLE "test_historicalchoice" (
    "id" integer NOT NULL,
    "poll_id" integer,
    "choice" varchar(200) NOT NULL,
    "votes" integer NOT NULL,
    "history_id" integer NOT NULL PRIMARY KEY,
    "history_date" datetime NOT NULL,
    "history_type" varchar(1) NOT NULL
)
;
CREATE TABLE "test_choice" (
    "id" integer NOT NULL PRIMARY KEY,
    "poll_id" integer NOT NULL REFERENCES "test_poll" ("id"),
    "choice" varchar(200) NOT NULL,
    "votes" integer NOT NULL
)
;
CREATE INDEX "test_historicalpoll_4a5fc416" ON "test_historicalpoll" ("id");
CREATE INDEX "test_historicalchoice_4a5fc416" ON "test_historicalchoice" ("id");
CREATE INDEX "test_historicalchoice_763e883" ON "test_historicalchoice" ("poll_id");
CREATE INDEX "test_choice_763e883" ON "test_choice" ("poll_id");
COMMIT;

* test was my app name for use in testing… poor choice.

The two historical db’s were created with the extra fields needed to track them historically:

  • test_historicalpoll
  • test_historicalchoice

Now to use, fire up the django shell $ ./manage.py syncdb and try this!

In [2]: from poll.models import Poll, Choice

In [3]: Poll.objects.all()
Out[3]: []

In [4]: import datetime

In [5]: p = Poll(question="what's up?", pub_date=datetime.datetime.now())

In [6]: p.save()

In [7]: p
Out[7]: <Poll: Poll object>

In [9]: p.history.all()
Out[9]: [<HistoricalPoll: Poll object as of 2010-10-25 18:03:29.855689>]

In [10]: p.pub_date = datetime.datetime(2007,4,1,0,0)

In [11]: p.save()

In [13]: p.history.all()
Out[13]: [<HistoricalPoll: Poll object as of 2010-10-25 18:04:13.814128>, <HistoricalPoll: Poll object as of 2010-10-25 18:03:29.855689>]

In [14]: p.choice_set.create(choice='Not Much', votes=0)
Out[14]: <Choice: Choice object>

In [15]: p.choice_set.create(choice='The sky', votes=0)
Out[15]: <Choice: Choice object>

In [16]: c = p.choice_set.create(choice='Just hacking again', votes=0)

In [17]: c.poll
Out[17]: <Poll: Poll object>

In [19]: c.history.all()
Out[19]: [<HistoricalChoice: Choice object as of 2010-10-25 18:05:30.160595>]

In [20]: Choice.history
Out[20]: <simple_history.manager.HistoryManager object at 0x1cc4290>

In [21]: Choice.history.all()
Out[21]: [<HistoricalChoice: Choice object as of 2010-10-25 18:05:30.160595>, <HistoricalChoice: Choice object as of 2010-10-25 18:05:12.183340>, <HistoricalChoice: Choice object as of 2010-10-25 18:04:59.047351>]

The highlighted lines show how to access the history of each object:

  • the first and second being the specific history of an instance (p – Poll, c – Choice)
  • the third is the history of an entire model (Choice)

Please leave a comment if you have any questions about this, or drop a bug report on bitbucket. I’m open to suggestions and/or more help to fill out some of the other relational field types (ManyToMany).

tl;dr – Use django-simple-history, it’s epic.

Continue reading » · Written on: 10-26-10 · 7 Comments »

give me a vision to believe in…

Not wanting to be left out in the cold, I took Monday off along with all the other lucky souls who were giving it off to relax and enjoy some R&R. Despite that, early on in the day I got pulled into helping a friend with some time critical work. The better part of my afternoon and night was spent doing research and testing out different technologies that really sparked a fire in me. It brought me back to place I had almost forgot existed; where work becomes fun. It didn’t even matter that I wasn’t getting paid or that it was my day off, I truly enjoyed tackling the problem and planning out / implementing solutions. Throughout the day and even now the only thought in my head is “why cant I make that a career?”

When I think back to early 2009 when I put this site up, I remember how hopeful I was at the prospect of a fresh start in my career. I was spending my days happily coding away on whatever project I pleased producing really awesome tools and generally getting back to the roots of my love for all things tech. I cant remember a time when I was happier (or poorer). In my bliss I imagined that ‘this could be every day of my life’. Work with some really cool tech with the freedom to explore, learn, and succeed all on my own? Yes please.

Then the real world caught up with me.

It’s a shame how life works out sometimes, but we take it in stride and push forward. I’ve fallen into a job that had a lot of similarities to the last job I had, yet at the same time is still very different. The people I have the pleasure of working directly with are truly passionate about the work they do day in an day out. As refreshing as it was to see my coworkers seemly enjoy their work, it made me realize that my love for that type of task was quickly fading. Today, I can feel those times in the days when work becomes work. To be honest, I cant remember a day where the office gave off any sense of drive other than emergency issues which really just equates to enormous amounts of stress. I hear the comments like “you should find new problems to keep yourself interested” or “self motivation is key” and I just shake my head. “What world do they live in?” is all I can think in response. I have all the negatives of having someone above me in a management roll without any of those really important perks.

  • Yes my hours are somewhat flexible.
  • No I cant spend a few hours to make system A instantly more useful.
  • Yes I’m not overly micro-manged.
  • No we cant just get something done, we need to play the office politics game and drag this out _years_.

“Oh but ____ is shielding you from _____.”
*rant* Really? Did I ask for that? Do I need that? I’m a big boy, I can handle myself. I don’t need you to speak for me. In most cases, I don’t want you to. How about you ask me what my thoughts on a subject are? How about we talk at all. In a year and a half, I’ve never really had a one on one with my manager. I’ve never really had more than a handful of three minute group discussions with them. I’m not an incapable human being; I can produce and clearly present well informed opinions and ideas. Please, if you must sit on that tier above me in the or chart and insist of directing my work, do me the kindness of talking to me. Thank you. */rant*

Every day is a constant reminder as to how unimportant we (the employees) are to my company. As I suspect it is with most companies, I generally feel that the higher-ups do not care one way or the other about anyone below them and to some extent I suspect many of them make the majority of their decisions based on how it helps them personally, not their people. My belief is that my current job is to develop tools and processes to make the lives of other employees easier so that they can work more effectively. If I do my job well everyone prospers all the way up the chain. I could do this in a number of places, but at this point I go into meetings (not that we have any) with suggestions that I’m already expecting to get shot down. It’s depressing and over the course of my tenure has inspired a level of cynicism that I didn’t know I was capable of.

I’m not oblivious to the fact that this is most likely what I can expect this from a lot of places. I believe our entire industry is plagued with poor or just outright bad management personnel and practices. I know I’m not alone in saying that there is a growing subdivision of young companies that are challenging the way the workplace looks and operates. I look and see these other companies; their team building events and employee benefits, their general ‘do good’ attitudes and well meaning company visions that give their people a reason to want to go to work every day. They recognize that they’re asking someone to dedicate a part of their life for the companies success; in turn, if it works out they take care of their people. Everyone wants the company to succeed. It’s no longer ‘just a job’, it’s what they want to be doing every day.

I want that.

ps: if this post doesn’t say have ‘career suicide’ stamped all over it, i don’t know what will.

Continue reading » · Written on: 10-12-10 · No Comments »

here, take these django-piston decorators

I’ve been using Jesper Noher’s django-piston project for some projects at work and home and I must say, I _really_ love it. Sadly, I don’t think the project gets as much credit as it should. If you’re looking to add REST to django, you really cant go wrong with piston.

That said, as I started to use it more and more, I hit some roadblocks that I had to come up with solutions to.

TLDR; Checkout my additions to django-piston here http://hg.qr7.com/django-piston/.

The @validate decorator in piston.utils worked great for creating new objects (POST), but didn’t really work at all when trying to update models (PUT). I extended this function to accept a model reference and define which fields are passed in through the url for existing object lookup. Yea, thats not clear, lets throw together an example (figure out your own imports). ***I haven’t tested this at all… let me know if i screwed up somewhere.

The model (models.py:

class MyModel(models.Model):
    testfield = models.CharField(max_length=50, unique=True)

And a form for validation (forms.py):

class MyModelForm(forms.ModelForm):
    class Meta:
        model = MyModel

Cant work without urls’s (urls.py):

basic_auth = HttpBasicAuthentication(realm='My Realm')
basic_mymodel = Resource(handler=MyModelHandler, authentication=basic_auth)
urlpatterns = patterns('',
    url(r'mymodel/$', basic_mymodel),
    url(r'mymodel/(?P<testfield>[^/]+)/$', basic_mymodel),
)

Finally, the handler (handlers.py):

class MyModelHandler(BaseHandler):
    """
    Authenticated entrypoint for MyModel entries.
    """
    model = MyModel
    anonymous = 'AnonymousMyModelHandler'
    fields = ('testfield',)
    allowed_methods = ('GET', 'POST', 'PUT')

    def read(self, request, *args, **kwargs):
        base = self.model.objects.all()
        if args != () or kwargs != {}:
            try:
                return base.get(*args, **kwargs)
            except self.model.DoesNotExist:
                return rc.NOT_FOUND
            except self.model.MultipleObjectsReturned:
                return rc.BAD_REQUEST
        else:
            return base

    @no_params()
    @validate(MyModelForm, 'POST')
    def create(self, request, *args, **kwargs):
        if not hasattr(request, "data"):
            request.data = request.POST
        attrs = self.flatten_dict(request.data)
        try:
            mymodel = self.model(testfield=attrs['testfield'])
        except:
            return rc.BAD_REQUEST
        else:
            mymodel.save()
        return mymodel

    @essential(['testfield'])
    @validate(MyModelForm, 'PUT', ['testfield'], MyModel)
    def update(self, request, *args, **kwargs):
        if not hasattr(request, "data"):
            request.data = request.POST
        try:
            instance = self.queryset(request).get(fqdn=kwargs['mymodel'])
        except self.model.DoesNotExist:
            return rc.NOT_FOUND
        except self.model.MultipleObjectsReturned:
            return rc.BAD_REQUEST
        except:
            return rc.BAD_REQUEST
        attrs = self.flatten_dict(request.data)
        for k,v in attrs.iteritems():
            setattr(instance, k, v)

        instance.save()
        return instance

As you can see on the highlighted lines, I’ve modified @validate to accept more params, and added the @no_params and @essential decorators. The three break down like this:

  • @validate now takes in an optional list of lookup fields, these should map to your urls.py as the inputs. In the example i have
  • @no_params is used to ensure that no parameters have been passed in, in this example, I want only POST’s to mymodel/ to be accepted for object creation. Without this, sometimes @validate will return weird errors that you I didn’t want the user to see. Doing the check in the actual create function wouldn’t work because @validate wraps it first and takes precedence.
  • @essential is used to ensure that the passed in list of parameters have been passed in to the request. In this case, testfield is essential for the existing object lookup (ok ok this is a bad example, I’m modifying a unique identifier, but thanks to django’s form validation, the new testfield should be checked for uniqueness as well!). Once we’ve ensured that testfield exists as a param, we can then validate against the form. I guess this is somewhat of a bad example, as the form will bitch that testfield is needed, but there are cases where the form bitches without any helpful information. I like having the option of presenting bad requests to users before the form displays errors. This serves that purpose very well.

All of this work was pretty straight forward. I went ahead an forked django-piston and made my changes to utils.py. You can find it all here: http://hg.qr7.com/django-piston/. I’ve issued a pull request to trunk and hopefully some/all of these will get added with any hope. If not, there here if you find yourself bumping up against the same problems. I’d be interested in anyones thoughts/contributions on all of this. Please drop me an email, comment, or PM me on bitbucket/twitter/wherever.

Hope this helps!

Continue reading » · Written on: 01-10-10 · 1 Comment »

Take my money FreeBSD… plox?

A brief story about OSS foundations, money, and why I hope FreeBSD dies a quick death.

Let me start by saying: every day I have to deal with FreeBSD. Through an unfortunate combination of circumstances, a co-worker has been blinded by the lies of FreeBSD and now swears by its superiority over other operating systems. Thus most of our servers that have been installed by him in recent years run the shit. Since I started this has been the subject of many day-to-day jokes/jabs in our small part of the floor.

Don’t get me wrong here; I know FreeBSD works on servers. I know it works well. However, I think FreeBSD is about as useful on the desktop as any Linux distro is (‘what a POS’). To this end, Gordon (a coworker) decided to surprise me with the following (which I only received after the fact):

Gordon wrote:

Hello FreeBSD Foundation,

I would like to make two donations. One from myself and one from a
co-worker. My donation isn’t a problem. However, the one for my
co-worker might be a problem. The donation from the co-worker has
to appear from him — but I would be paying for it.

(It’s a bit of a practical joke on my part. The coworker doesn’t
much care for FreeBSD. He’s one of those filthy Debian users. Seeing
his name appear as a donor for FreeBSD would result in much comedy at work)

Can this be arranged?

Regards,

-Gordon

Son of a bitch.

This f’n guy. Do you see what thisĀ  guy is doing to me? KILLING ME SLOWLY.

_Well_Fucking_Played_Gordon_

So this letter gets sent to deb@freebsdfoundation and board@freebsdfoundation. One would think that Gordon’s going to slay with this one, right? WRONG.

Instead of taking the free money, although slightly underhanded, in pretty good faith / kind gesture, the FreeBSD foundation replies to him with this:

Dear Gordon,

Thank for your interest in making a donation to the foundation. I
understand your intent on playing a practical joke on your co-worker.
We all try to have a good sense of humor here. But, our donors list is
extremely important to us. This is where we have a chance to acknowledge
and show our appreciation to our donors. The names listed there are
FreeBSD’s biggest advocates. And, BTW, we do have a policy that we only
list the name as documented on the check or donation service.

I hope you understand why we can’t support you with your request.
Thank you for your support of FreeBSD.

Sincerely,

Deb Goodkin
Director of Operations
The FreeBSD Foundation

Wut?

Really?

_Really?_

Listen guys. FreeBSD isn’t so hot nowadays. When someone offers you money for your shitty work, take it. Take it and fucking run. Use it to get that stick shoved up your ass surgically removed. What if he was going to donate hundreds of dollars? You wont add 12 letters to a web page because someone might be butthurt if they found out the true story behind it? This is whats wrong with these shitty fucking OSS projects.

For future reference FreeBSD foundation: NO ONE GIVES A FUCK.

Idiots.

Anyway, I will be making a donation to some shitty OSS project other than the FreeBSD project and forwarding the receipt on to Deb in reply to her message in the near future. Thanks for the laughs.

Continue reading » · Written on: 10-28-09 · 1 Comment »