Professional Documents
Culture Documents
A Summer in The Wild
A Summer in The Wild
A Summer in The Wild
Alex Gaynor
DjangoCon.us 2011
6 September 2011
Alex Gaynor
Alex Gaynor
Me
Django/CPython/PyPy core developer Mostly been building websites for the past 4 years or so. I was getting kind of sick of websites.
Alex Gaynor
PyPy
Implementation of Python, written in [a restricted subset of] Python Advanced tracing just-in-time compiler. Fast: http://speed.pypy.org/
Alex Gaynor
Quora
Question and answer, knowledge database site All Python Wanted it to go faster.
Alex Gaynor
This Summer
Alex Gaynor
This Talk
Everything I learned in 4 months... in 30 minutes. Stuff I missed terribly when I wasnt using Django Stuff you need to be doing, on all your projects.
Alex Gaynor
First Mover
Quora was (to my knowledge) the largest website to migrate to PyPy Not all packages worked out of the box
Alex Gaynor
MySQLdb
Wrote a fresh implementation of MySQLdb using ctypes Knew it was going to be needed going in. Still a pain, next guy doesnt have to do that.
Alex Gaynor
MYSQL RAGE
32-bit vs. 64-bit Client has been blocked from this server
Alex Gaynor
Paster
Under CPython used pasters HTTP server It had obscure bugs under PyPy with chunked encoding Switched to gunicorn: Just Worked (tm)
Alex Gaynor
Cython
Division between .py and .pxd les Better support hopefully coming soon.
Alex Gaynor
On being rst
Any major infrastructure change carries inherent risk. Know what that risk is going in. Tests are an absolute must (more on this later).
Alex Gaynor
Proling
It sucks. siege for very basic comparison. Editing gunicorn itself for actual proling with cProle (wtf?)
Alex Gaynor
Alex Gaynor
URLs
With 100% ease, I can start from the root URL conf, and track down what view serves a given request. DO NOT VIOLATE THIS WITH MIDDLEWARE HACKS.
Alex Gaynor
The ORM
You forget how awesome it is, until you dont have it. 80-20 solution, more like 95-5 solution.
Alex Gaynor
Escaping
Security as a top priority. You have to actively work at it to give yourself an XSS hole.
Alex Gaynor
Conventions
You always know where your models, views, urls, etc. are within an app.
Doesnt require Django itself to do anything.
We need to keep going with this, organize projects in the same way. Conventions for using WSGI middleware with Django (coming soon)
Alex Gaynor
Seperation of concerns
Alex Gaynor
Alex Gaynor
Project startup
cvs checkout mkvirtualenv <project> pip install -r <path/to/requirements.txt> Edit some settings.py
We need better conventions for settings.
We do not
Check python binaries into our VCS Check compiled modules into our VCS Vendor our dependencies. Run your own PyPI server if you feel the need, dont check crap into the repository. Have a process that isnt a good process. Just a process isnt enough.
Alex Gaynor
Automate Everything
Content on Quora automatically updates if its changed by anyone on the site. General framework for doing this, not done manually. (Cool idea, someone should write it for Django) To a rst approximation anything done manually will break.
Alex Gaynor
Automate EVERYTHING
DRY applies to the human side of things as well. Write tests so you dont need to test everything by hand when you make a change Have a buildserver to run tests One-click deploys If you run the same 3 commands for occasional maintenance, PUT THEM IN A SHELL SCRIPT
Alex Gaynor
Caching
At a minimum organized. Better is automated (easier for simple object-caches). No caching in your views.
Alex Gaynor
Tests
How do you know what youre writing works? How do you know you didnt just break it? How do you know your big refactor still does the right thing?
Alex Gaynor
Build server
Because I dont trust myself, much less everyone else, to run all the tests every time
Alex Gaynor
{fabric, gondor, ep.io, chef, puppet} deploy git push deploy WHATEVER Deployment is hard enough, without having to re-invent it each time.
Alex Gaynor
In Summary
Infrastructure changes can be dangerous, mitigate risk by doing research up front Django is lovely, take advantage. Automate everything, or you and your colleagues will slowly go mad.
Alex Gaynor
Questions?
@alex_gaynor
http://alexgaynor.net/
Alex Gaynor