I’m back. For real this time.
Quick up date: I work in NYC now building out lofty.com. Life has been chaos as usual.
Coming up:
Eight months! Apparently that is the magic number when it comes to the amount of time it takes for me to talk about work I’ve done and get motivated to polish it up a bit. That said, lets talk about django-simple-history! One of my bigger projects at work is a monstrous django project that I’ve built from the ground up that does a bunch of enterprise type nitty-gritty (yay CRUD). At time of inception, one of the big feature requests was to have tracking of _all_ changes to the database. This request included needs for the following:
I looked around and saw some basic implementations of different historical record attempts, including AuditTrail but had a requirement of needing models.OneToOneField & models.ForeignKey fields to work (store that there was a relation and not fail on lookups). I had read about something similar in Marty Alchin’s Pro Django book (highly recommend), so I dug that up and was sad to see that it also didn’t work with the relation fields. I reached out to Marty to see if he had a solution and it seemed that at the time he had not even thought about it too much past his initial work. I asked him if he’d be OK with me expanding upon his initial source and creating a project out of his work with changes to support the relation fields and he agreed. The one aspect that had to be left out for now is the tracking of the user who made the change. This _can_ be done. I do it with an extra field added to the history model. That said, hopefully soon I can release that code with a patch to django-simple-history so you can easily do this. For now please contact me for more details about tracking the current user making changes. That said, everything else was free game so I threw it all together and put it on bitbucket.
Thus django-simple-history was born.
I kinda threw the code up after i got it working and totally forgot about it. I’m sad to admit that I didn’t do it due diligence, even if I never planned to make it a package. That was until someone actually forked my code! *SHOCK* They had added a setup.py, so I pulled it in and expanded on it (it was pretty simple at first). Now there is a fully installable app with example code in the README and an actual package uploaded to make django-simple-history 1.0!
Here’s the basic rundown on how you use it using the django tutorials Poll and Choice models (used django 1.2.3 for these tests):
from django.db import models
# import the HistoricalRecords model
from simple_history.models import HistoricalRecords
class Poll(models.Model):
question = models.CharField(max_length = 200)
pub_date = models.DateTimeField('date published')
# create an instance of HistoricalRecords on any model you want to track
history = HistoricalRecords()
class Choice(models.Model):
poll = models.ForeignKey(Poll)
choice = models.CharField(max_length=200)
votes = models.IntegerField()
history = HistoricalRecords()
That’s it! Once you’ve done this, when you $ ./manage.py syncdb you will get the following:
BEGIN;
CREATE TABLE "test_historicalpoll" (
"id" integer NOT NULL,
"question" varchar(200) NOT NULL,
"pub_date" datetime NOT NULL,
"history_id" integer NOT NULL PRIMARY KEY,
"history_date" datetime NOT NULL,
"history_type" varchar(1) NOT NULL
)
;
CREATE TABLE "test_poll" (
"id" integer NOT NULL PRIMARY KEY,
"question" varchar(200) NOT NULL,
"pub_date" datetime NOT NULL
)
;
CREATE TABLE "test_historicalchoice" (
"id" integer NOT NULL,
"poll_id" integer,
"choice" varchar(200) NOT NULL,
"votes" integer NOT NULL,
"history_id" integer NOT NULL PRIMARY KEY,
"history_date" datetime NOT NULL,
"history_type" varchar(1) NOT NULL
)
;
CREATE TABLE "test_choice" (
"id" integer NOT NULL PRIMARY KEY,
"poll_id" integer NOT NULL REFERENCES "test_poll" ("id"),
"choice" varchar(200) NOT NULL,
"votes" integer NOT NULL
)
;
CREATE INDEX "test_historicalpoll_4a5fc416" ON "test_historicalpoll" ("id");
CREATE INDEX "test_historicalchoice_4a5fc416" ON "test_historicalchoice" ("id");
CREATE INDEX "test_historicalchoice_763e883" ON "test_historicalchoice" ("poll_id");
CREATE INDEX "test_choice_763e883" ON "test_choice" ("poll_id");
COMMIT;
* test was my app name for use in testing… poor choice.
The two historical db’s were created with the extra fields needed to track them historically:
Now to use, fire up the django shell $ ./manage.py syncdb and try this!
In [2]: from poll.models import Poll, Choice In [3]: Poll.objects.all() Out[3]: [] In [4]: import datetime In [5]: p = Poll(question="what's up?", pub_date=datetime.datetime.now()) In [6]: p.save() In [7]: p Out[7]: <Poll: Poll object> In [9]: p.history.all() Out[9]: [<HistoricalPoll: Poll object as of 2010-10-25 18:03:29.855689>] In [10]: p.pub_date = datetime.datetime(2007,4,1,0,0) In [11]: p.save() In [13]: p.history.all() Out[13]: [<HistoricalPoll: Poll object as of 2010-10-25 18:04:13.814128>, <HistoricalPoll: Poll object as of 2010-10-25 18:03:29.855689>] In [14]: p.choice_set.create(choice='Not Much', votes=0) Out[14]: <Choice: Choice object> In [15]: p.choice_set.create(choice='The sky', votes=0) Out[15]: <Choice: Choice object> In [16]: c = p.choice_set.create(choice='Just hacking again', votes=0) In [17]: c.poll Out[17]: <Poll: Poll object> In [19]: c.history.all() Out[19]: [<HistoricalChoice: Choice object as of 2010-10-25 18:05:30.160595>] In [20]: Choice.history Out[20]: <simple_history.manager.HistoryManager object at 0x1cc4290> In [21]: Choice.history.all() Out[21]: [<HistoricalChoice: Choice object as of 2010-10-25 18:05:30.160595>, <HistoricalChoice: Choice object as of 2010-10-25 18:05:12.183340>, <HistoricalChoice: Choice object as of 2010-10-25 18:04:59.047351>]
The highlighted lines show how to access the history of each object:
Please leave a comment if you have any questions about this, or drop a bug report on bitbucket. I’m open to suggestions and/or more help to fill out some of the other relational field types (ManyToMany).
tl;dr – Use django-simple-history, it’s epic.
I’ve been using Jesper Noher’s django-piston project for some projects at work and home and I must say, I _really_ love it. Sadly, I don’t think the project gets as much credit as it should. If you’re looking to add REST to django, you really cant go wrong with piston.
That said, as I started to use it more and more, I hit some roadblocks that I had to come up with solutions to.
TLDR; Checkout my additions to django-piston here http://hg.qr7.com/django-piston/.
The @validate decorator in piston.utils worked great for creating new objects (POST), but didn’t really work at all when trying to update models (PUT). I extended this function to accept a model reference and define which fields are passed in through the url for existing object lookup. Yea, thats not clear, lets throw together an example (figure out your own imports). ***I haven’t tested this at all… let me know if i screwed up somewhere.
The model (models.py:
class MyModel(models.Model):
testfield = models.CharField(max_length=50, unique=True)
And a form for validation (forms.py):
class MyModelForm(forms.ModelForm):
class Meta:
model = MyModel
Cant work without urls’s (urls.py):
basic_auth = HttpBasicAuthentication(realm='My Realm')
basic_mymodel = Resource(handler=MyModelHandler, authentication=basic_auth)
urlpatterns = patterns('',
url(r'mymodel/$', basic_mymodel),
url(r'mymodel/(?P<testfield>[^/]+)/$', basic_mymodel),
)
Finally, the handler (handlers.py):
class MyModelHandler(BaseHandler):
"""
Authenticated entrypoint for MyModel entries.
"""
model = MyModel
anonymous = 'AnonymousMyModelHandler'
fields = ('testfield',)
allowed_methods = ('GET', 'POST', 'PUT')
def read(self, request, *args, **kwargs):
base = self.model.objects.all()
if args != () or kwargs != {}:
try:
return base.get(*args, **kwargs)
except self.model.DoesNotExist:
return rc.NOT_FOUND
except self.model.MultipleObjectsReturned:
return rc.BAD_REQUEST
else:
return base
@no_params()
@validate(MyModelForm, 'POST')
def create(self, request, *args, **kwargs):
if not hasattr(request, "data"):
request.data = request.POST
attrs = self.flatten_dict(request.data)
try:
mymodel = self.model(testfield=attrs['testfield'])
except:
return rc.BAD_REQUEST
else:
mymodel.save()
return mymodel
@essential(['testfield'])
@validate(MyModelForm, 'PUT', ['testfield'], MyModel)
def update(self, request, *args, **kwargs):
if not hasattr(request, "data"):
request.data = request.POST
try:
instance = self.queryset(request).get(fqdn=kwargs['mymodel'])
except self.model.DoesNotExist:
return rc.NOT_FOUND
except self.model.MultipleObjectsReturned:
return rc.BAD_REQUEST
except:
return rc.BAD_REQUEST
attrs = self.flatten_dict(request.data)
for k,v in attrs.iteritems():
setattr(instance, k, v)
instance.save()
return instance
As you can see on the highlighted lines, I’ve modified @validate to accept more params, and added the @no_params and @essential decorators. The three break down like this:
@validate now takes in an optional list of lookup fields, these should map to your urls.py as the inputs. In the example i have @no_params is used to ensure that no parameters have been passed in, in this example, I want only POST’s to mymodel/ to be accepted for object creation. Without this, sometimes @validate will return weird errors that you I didn’t want the user to see. Doing the check in the actual create function wouldn’t work because @validate wraps it first and takes precedence.@essential is used to ensure that the passed in list of parameters have been passed in to the request. In this case, testfield is essential for the existing object lookup (ok ok this is a bad example, I’m modifying a unique identifier, but thanks to django’s form validation, the new testfield should be checked for uniqueness as well!). Once we’ve ensured that testfield exists as a param, we can then validate against the form. I guess this is somewhat of a bad example, as the form will bitch that testfield is needed, but there are cases where the form bitches without any helpful information. I like having the option of presenting bad requests to users before the form displays errors. This serves that purpose very well.All of this work was pretty straight forward. I went ahead an forked django-piston and made my changes to utils.py. You can find it all here: http://hg.qr7.com/django-piston/. I’ve issued a pull request to trunk and hopefully some/all of these will get added with any hope. If not, there here if you find yourself bumping up against the same problems. I’d be interested in anyones thoughts/contributions on all of this. Please drop me an email, comment, or PM me on bitbucket/twitter/wherever.
Hope this helps!
Or how to “crack” super fast WEP in < 1 minute.
I’ve been known to “borrow” internet from time to time ::shifty eyes:: and my recent moves have been no different. The place I was at last month was served by Comcast so getting online was a breeze with some fancy tools, but since then I’ve now moved to a COX neighborhood and the amount of work involved has proven to be much more involved than original anticipated.
For the time being, I’m stuck waiting for COX to come out and plug in some stuff; without getting into too many details lets just say I didn’t feel like being a jerk to the techs and giving everyone on the block free TV/cable internet. More on this in a later post…
For now I was back to the old fashion wireless internet “auditing”. The nice thing about this location is the sheer number of people with some really nice high speed connections. I spent half a day blowing through as many AP’s as I could only to continuously stumble upon these 5 character long SSID’s that ended up belonging to FIOS users. Whats worse is the 64 bit WEP keys they where using where pretty similar too. Check it out:
9FRV0 : '1801169C95'
9YCQ3 : '1F905FA711'
A8GP8 : '1F90DF266A'
GV062 : '1801378BEC'
Q0BL2 : '1801426D0A'
*there where about 15 more just like these
Notice anything? Here’s a hint, the first 4 characters of the WEP key appear to be common with two variations, ’1801′ and ’1F90′, appearing in this sample set. These characters match up to the 2-6th wireless MAC address octets from the AP itself.
Now, you could capture some packets with aircrack and do something like:
aircrack-ng -d XXXX capturefile.cap
where XXX is the __:XX:XX:__:__:__ portion of the access points mac, but who has the time for that shit anymore 
At this point you should be appalled. This is security at its very worst. These users think they’re secure; as if it wasn’t bad enough that verizon is using WEP by default, a well known hackable wireless ‘encryption’. Lets delve a little deeper…
A quick google search led to someone mentioning that if you Base36 the SSID, convert it to hex, and throw that after the four characters from the MAC, it should equal the key.
Well then lets try! Let’s just use the ‘9FRV9‘ SSID from above to see if this really works.
I needed a base36 converter so I went here and did the following:
base36 of ssid: 15852492
base36 results in hex: F1E3CC
Wait wait wait, thats not right. The unknown part of our key should be ‘169C95‘!
After reading up on base36 I decided to try it myself and see what happens:
In [2]: b36d('9FRV0')
Out[2]: 1481877
In [3]: hex(1481877)
Out[3]: '0x169c95'
Well now that’s more like it!
So why the difference in my b36d function? Turns out this is not how you do base36! My implementation of base36 wasn’t reversing the string, and it appears I’m not the only one making this mistake ::cough::Verizon::cough::. That said, mine worked, so who cares.
So now giving the mac structure of: __:18:01:__:__:__ we can guess the key as: 1801169c95
Pretty cool stuff. I decided this was useful enough to automate so I threw together a quick script and its now up in a repo on my bitbucket. Check it out here: http://bitbucket.org/q/verizon_wepkey/
I’m not going to internet white knight it and tell you not to use this information in a bad way; I’m not that naive. What I do want to stress is that you secure the networks of those you can help, and maybe lend a neighborly helping hand if you see this going on.
Verizon needs to seriously reconsider their actions here and re-evaluate how they handle their customers. Right now its a free for all on fiber internet, and eventually someones going to get burned by this.
It’s been a long month, and there just doesn’t seem to be an end in sight! I’ve been living a more fulfilling life for the last 2 months than I did for the previous 1.5 years. My old job consumed me; I brought work home every night, and that was only after I had been at work well past everyone else had left. Don’t get me wrong, it was my choice, and I felt like I had nothing but work so I didn’t mind it at the time. Looking back now however, I’m glad it’s over, and I feel no regret towards taking the last two months to realize who I am and what I love.
I sent myself to pycon at the end of March as a mini ‘vacation’ of sorts (despite the fact that I can’t really afford to spend money like that). The trip started off rough with me missing my flight out, but picked up from there. Truthfully, I had a great time. I didn’t really learn much, but I did get the chance to meet some really cool guys, got to eat lunch with python superheroes and generally just enjoyed myself for a few days away from DC.
I spent the week after pycon packing my stuff up, moving it all to NoVA, and cleaning the old house. Truly an awful experience. I either need to get rid of my expensive stuff, or just generally stop buying toys. I have a spending problem, I admit it. I didn’t really need 5 turntables, or 7 rack servers, or 6 UPS (which weigh a f’n ton) just to name a few things off the top of my head.
I got a chance to hang out with some local security guys, and as an added benefit I got a feel for some of the jobs out there as well. Finished up an interview process with a company I think I could do really great things at. Got a chance to check out NoVA-DUG (really enjoy this lot!) and it inspired me to throw all of that useless Django knowledge I’ve got floating around in my fat head into an app. Don’t have a name for it now, but it’s meant to be a middle ground between VCS’s and those groups out there that just haven’t gotten on board with storing code in repo’s (I know they’re out there, I recently worked at one). Did most of the work last weekend and have spent a few hours here and there getting up to speed on jQuery to give the site that o-so-loved web2.0 look/feel. FunFact: I am terrible at web design!
This morning I got an offer letter that I need to respond to within a week. I actually like the job a lot on paper. I feel like I’d be better supported than past experience and from the looks of it I could actually walk to work in minutes! That said, I’m apprehensive about this decision and want to make sure I really weigh out my options. The way I put it: I don’t want to look back six months from now and wish I had made a different decision. I wont stick around at a job that makes me miserable again, but at the end of the day I hate saying ‘no’ to people too. I would drag it out and just make myself miserable and it wouldn’t be right. Wherever I do end up I want to feel really good about, and hopefully I’ll have a better idea of where that place is within the week. The last of my resumes have been sent to the appropriate parties and now I’m just waiting. I’ll post an update when I have a better idea of where I’m going to end up (if I can).
Last weekend I got word that my motorcycle has been fixed and is ready to go! First thought: F YEA! Second thought: FML, I don’t have a helmet. I had actually ordered a replacement the night before the call came (thinking it would still be another week till my bike was done). None the less, my helmet should be in on Thursday, and starting Friday the forecast for the next week is nothing but clear skies and 75-85*F. I’m going to try and get in as much riding as possible (while finding a job!).
