You’ve been doing small scripts for a while and you’ve started getting bigger and bigger projects. But you're getting the fear, the dreaded imposter syndrome. Now to be fair we all get it and we all have to start somewhere. So here are a few tips to get you from somebody who does Python to a Python developer.
Over the next weeks, I will be adding articles that cover each of these sections in depth with project layout examples and some example code.
Update: I’ve written an article about python development in teams, and more detailed best practices.
Coding Python in a team… of one or more
If you’re a beginner or even if you’ve been doing it for years it’s easy to get lazy and just get by, on doing what you…
Use a source control service
This should be a developer's Zeroth law, the one that comes before everything else. Your code is the output of your labour and as such you need to keep it safe. At the very minimum, source control like Git is a very sophisticated way to back up your code. But if you’re new to source control and the idea of branches, tags, merging and all that makes you break out into a cold sweat don’t worry. Most of the features in source control are designed around working in a team, so to begin with you won’t have to touch it. To start off, use an online git provider like GitLab, Bitbucket or GitHub and start small, with just checking in as you go. This will give you a way to roll back changes and see a history of what you did and when. If you’re doing blocks of work and then releasing you can move on to tags.
Once you start working in a team, this is where branches become very useful. By default, the code you check in will go onto the master branch. As you move onto splitting up work amongst different team members this is where a technique like feature branches will fit in. You should quickly get to a stage where no work is lost and you can track all the changes to your code.
Choose your editor to help you be lazy
As a developer, all you have is time and knowledge. There are no raw materials you have to hue into a new product, it all comes from your brain through your fingers and into the computer. So do everything you can to speed up your process, and keeping track of what things are called is a continual source of development treacle. Let your editor do the heavy lifting.
There are 2 hard problems in computer science: cache invalidation, naming things, and off-by-1 errors. — Leon Bambrick
Code completion and function lookup helpers are crucial in speeding up your development. Editors use class and type inference to provide pop up suggestions, parameter lists and even python documentation as you type. This function allows you as a developer to stay on track and in the flow without having to break out into looking up method calls in your code or breaking out the Google.
Choosing your favourite code editor can be a bit of a religious war among developers. Ignoring the vi and emacs debate/war that has been going on longer than the dawn of time, here is a list of the top slim-ish editors around.
- Visual Studio Code — My personal favourite and a relatively new kid on the block. Microsoft has made an excellent lightweight and feature-rich editor that is free in all senses of the word. Free
- Sublime Text — An excellent editor and one which was my primary editor for a very long time. It’s powerful and has great language support. $80
- PyCharm — An editor edging into full IDE territory with a long and loyal following. I haven’t used it since the early days. Worth a try. £69/$89 per year
Use unit and integration tests
Unit and integration tests are your safety net. If you can’t test your code, you can’t trust your code.
We all know, or at least should know that writing test code is something we should all be doing, but like all good things from healthy eating to regular exercise, it’s not that fun. We all want to get on to the next fun piece of work and spend less time writing tests, but it just needs to be done. Even when it comes to your own code you’re going to be a stranger to it in 6 months time. If you need to rewrite a validation routine 6 months down the line to handle a bunch of new circumstances, you need to know all the old test cases will still work as you tweak, and rehash your old regular expressions. It’s very normal for the test code for a Class to be much larger than the class itself.
It’s common for developers to say that “Writing tests takes up too much time.” or “My code doesn’t work in a way that can be tested like that.” Firstly, debugging takes time, deploying code with some edge cases takes time, manually checking everything works after a code update takes time. Tests save time. Secondly, if your code is not written in a way that can be easily tested it is bad code and needs to be rewritten.
Document your code
Even if you’re a lone developer, you need to document and test like you’re in a team.
If there’s one thing that developers hate more than writing tests, it’s writing documentation. As I’ve said before, you’re going to be a stranger to your code in 6 months time, so help your future self out by documenting.
There are two levels of documentation, the API docs that are generated from your inline comments, and the much more difficult detailed documentation that uses full sentences and mystical things like examples. If you’re not used to writing documentation get yourself started with baby steps and start writing proper python code comments. I use the Google style python docstrings as I find them to be the most aesthetically pleasing and there is good support in Sphinx for generating the HTML or PDF documentation. And use Sphinx! I know it looks crunchy and a bit archaic, but once it’s set up it just sort of works. Sphinx uses reStructured text templates for documentation and lots of editors now support live preview of the rendered view it will generate.
Script everything or CI/CD is your friend
If you do something once it’s an experiment. If you do it more than once it’s a script.
If you have a fancy pants continuous integration set up at your workplace then great, find out how to automate the CI/CD pipeline and automatically test everything on check-in. In my environment, I’ve set up a GitLab server that handles all the git repositories and uses the CI/CD features of GitLab to run automated tests every time code is committed to a repository. To start with this will probably be overkill, but as you start collaborating with other people on a code base you may want to look at running a full CI/CD pipeline.
One of the best things to script first is your environment setup. This can be a simple as creating a python virtual environment, installing all the python dependencies and maybe to be super slick, a script to run your unit tests. You have done some unit tests, haven’t you? Learning a bit of bash can be a superpower when it comes to automation, but keep them small, they are not your day job.
Second, is to create a script that runs your unit tests and third, a script that generates your documentation.
If your module, library or application relies on some large test files unsuitable for version control, then the generation/retrieval of that test data should be scripted. Within my professional capacity, I develop modules that analyse high-resolution image files. The environment setup script generates some images and retrieves some others from a web server using curl.