Create a requirements file with only top level dependencies
You can create a list of your dependencies with pip freeze
. However, if you look at the output, it's going to be obvious right away that something is not right:
$ pip install flask Collecting flask ... $ pip freeze click==6.6 Flask==0.11.1 itsdangerous==0.24 Jinja2==2.8 MarkupSafe==0.23 Werkzeug==0.11.11
Whoa. Yeah, I wanted the dependencies of flask to be installed for sure, but why would I want that in my requirements? Now I have to keep track of my depedencies' dependencies (and so on all the way down), because when I decide that I no longer need flask, I have to remove the dependencies of it manually. The requirements output actually looks like what you might find in a lockfile (like composer.lock for Composer, or shrinkwrap.json for NPM), not what you might find in your declared dependencies.
Pip-tools
To solve the above problem, we have to use a separate tool, for god knows what reason, because this should be the package manager's job, but it's not included in pip. So if you are also wondering about this, don't worry, that tool thankfully exists, and it's called pip-tools, you install it with the usual pip install pip-tools
. It introduces a real dependency file called requirements.in
and makes requirements.txt
the lock file, AS IT SHOULD HAVE BEEN ALL ALONG.
There are two commands it provides:
pip-compile [requirements-in]
This will read your requirements.in
file, and create a requirements.txt
, with the output that you would expect:
$ echo "flask" >> requirements.in $ pip-compile # # This file is autogenerated by pip-compile # To update, run: # # pip-compile --output-file requirements.txt requirements.in # click==6.6 # via flask flask==0.11.1 itsdangerous==0.24 # via flask Jinja2==2.8 # via flask MarkupSafe==0.23 # via jinja2 Werkzeug==0.11.11 # via flask
And boom, you are done, requirements.in
only contains your top level dependencies, and as a bonus your requirements.txt
now shows which package is required by another package.
pip-sync
This command will install all the packages in your requirements.txt, and remove any superflous packages that you have installed, but not recorded in your requirements. The difference between this, and the usual pip install -r requirements.txt
is the latter part. Since pip install -r
does not remove superflous packages, if you have forgotten to record something, you will not find out about it, until it blows up in some other environment. Instead of using pip install X
, you put X in your requirements.in file, and then do pip-compile && pip-sync
to actually install the package, avoiding the aforementioned problem.
As a bonus, updating your packages can be done with pip-compile -U
. This will take a look at dependencies, and look for updates for any package that has a flexible pinning (eg. >=1.0)
Dev requirements
Both composer and npm can record "dev" requirements, that is, when you need packages for development, but not for production. Pip doesn't have this feature explicitly, and it doesn't really advertise it, but it can actually do this. You can put a line with -r otherfile.txt
on the top of your requirements, and it will include another file with a list of packages. So to do a separation of dev libraries, you can have two files:
-r requirements.txt nose
flask
And then you use the requirements file that you need for the current environment. Pip-tools also supports this in the *.in
files.