Skip navigation

Category Archives: Technical Problems

Coming off from my last post about updating pip inside a virtualenv, I have, finally, realized what I’ve been doing wrong all this time: I use this nifty script called virtualenv burrito to get a virtualenv + virtualenvwrapper set-up.

I’ve been writing an install script for a new project I am working on. Being at such an early phase, the script is scrappy and will only probably work for machines configured exactly as mine. Not that I make a lot of customizations, but still it relies on the existence of virtualenvwrapper too much.

The thing is I could never get my install script to run properly since pip is broken in my virtualenvs. So just yesterday, I finally decided to devote some time to digging into why precisely this is happening.

Long story short, after a few hours digging around the Vogon poetry of bash scripting, I realized that I can’t find separate virtualenv and virtualenvwrapper installs in my system…because I used virtualenv burrito to install them.

So I deleted the .venvburrito directory in my home and rerun virtualenv burrito. And…tadddaaaahhh!

chad@scheherazade:~$ mkvirtualenv fresh
New python executable in /home/chad/.virtualenvs/fresh/bin/python
Installing setuptools, pip, wheel...done.
virtualenvwrapper.user_scripts creating /home/chad/.virtualenvs/fresh/bin/predeactivate
virtualenvwrapper.user_scripts creating /home/chad/.virtualenvs/fresh/bin/postdeactivate
virtualenvwrapper.user_scripts creating /home/chad/.virtualenvs/fresh/bin/preactivate
virtualenvwrapper.user_scripts creating /home/chad/.virtualenvs/fresh/bin/postactivate
virtualenvwrapper.user_scripts creating /home/chad/.virtualenvs/fresh/bin/get_env_details
(fresh) chad@scheherazade:~$ 
(fresh) chad@scheherazade:~$ pip -V
pip 10.0.1 from /home/chad/.venvburrito/lib/python2.7/site-packages/pip-10.0.1-py2.7.egg/pip (python 2.7

What this does not address, however, are my old virtualenvs which are stuck on an outdated pip. But I guess it should be trivial to just delete those virtualenvs and create a new one, this time with an updated pip.

There’s this thing in life, where you have assumptions you neglect to state because you thought (assume) them to be inconsequential, then they turn out to be of great importance. This is all the more true when you work with computers.

I love Python. I think it is a very productive language for prototyping, pedagogy, and experimentation with a mature ecosystem to support you as you take your shit code ideas to production. However, in all my years using it, I could never get to upgrade pip, its preferred package manager, inside a virtualenv without problems. And that frustrates me.

The story starts with one of the first things you tell Python noobs before they pwn their own system and blame it all on hallowed Python: use virtualenvs. I’m not sure if this applies to Windows users (or if the concept of virtualenvs even applies to Windows users) but for *nix users, this advice makes sense since Python is often used by your system for other, more-important-than-your-side-projects, OS-y things it does. This is why Python often comes pre-packaged with the system and why you should use virtualenvs: the last thing you want is to upgrade the datetime library which your NTP daemon is using only to find out that said daemon relies on a particular quirk of that datetime library version to function properly. And so virtualenvs.

I do not intend to discuss how virtualenvs work, but suffice it to say that it does magic so that the libraries your projects need are isolated. If that whole story about NTP daemons and datetime above just flew over your head, at least appreciate how nice it is to have your dependencies isolated like that; alexandria could not see PyGame, and PyGame Objects could not see Flask, and yet I develop them on the same machine. It tingles that part of my brain which makes me put my glass/cup down at different spots on the table every time after I take a sip so that I end up with liquid circles touching each other but not overlapping. An ordered world is just so zen.

(Although I should note that I’ve long given up making PyGame work consistently with virtualenvs across the machines I use. Nowadays, I just containerize the whole thing to find almost the same peace of mind.)

Anyway, I’m beginning to digress. To proceed with the story, there comes a time when issuing pip commands inside a virtualenv results to the following warning:

(origin3) chad@scheherazade:~$ pip freeze
You are using pip version 8.1.2, however version 10.0.1 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.

And, well, who does not want free updates? Not to mention when the software involved is not just free but also open-source. And have I told you that this free and open-source software is part of the ecosystem of my favorite programming language? Up-update and away, I say!

(origin3) chad@scheherazade:~$ pip install --upgrade pip
Collecting pip
  Using cached
Installing collected packages: pip
  Found existing installation: pip 8.1.2
    Not uninstalling pip at /home/chad/.venvburrito/lib/python2.7/site-packages/pip-8.1.2-py2.7.egg, outside environment /home/chad/.virtualenvs/origin3
Successfully installed pip-8.1.2
You are using pip version 8.1.2, however version 10.0.1 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.

Err…okay, that did not really work. But not to worry! The beauty of using FOSS is that help is almost always available online. Especially on StackOverflow, a pillar of modern software engineering. Which leads us to the following words of wisdom:

pip is just a PyPI package like any other; you could use it to upgrade itself the same way you would upgrade any package:

~Cairnarvon on StackOverflow

But, but…Cairnarvon good sir, said instructions not working is what exactly set me on this quest! Then again, no one else is complaining that the accepted answer does not really work so I guess I’m the idiot here.

Thankfully, there are more words of wisdom in StackOverflow. If one option does not work, you have options. Fortunate then that this idiot can read. Since “pip is just a PyPI package”, it stands to reason that you can also upgrade it using its totally-not-sexy sibling of a package manager, easy_install.

Now, the other thing you tell Python noobs, maybe even before you tell them about virtualenvs, is to use pip over easy_install because reasons. So at this point, I am conflicted: one of the most ingrained teachings in my being is to use pip over easy_install but I have also been told to use updated software packages as much as possible. The latter urge wins out so I end up doing naughty things…

(origin3) chad@scheherazade:~$ easy_install pip
Searching for pip
Best match: pip 10.0.1
Adding pip 10.0.1 to easy-install.pth file
Installing pip3.6 script to /home/chad/.virtualenvs/origin3/bin
Installing pip3 script to /home/chad/.virtualenvs/origin3/bin
Installing pip script to /home/chad/.virtualenvs/origin3/bin

Using /home/chad/.virtualenvs/origin3/lib/python3.5/site-packages
Processing dependencies for pip
Finished processing dependencies for pip

…with good results:

(origin3) chad@scheherazade:pydagogical$ pip --version
pip 10.0.1 from /home/chad/.virtualenvs/origin3/lib/python3.5/site-packages/pip (python 3.5)

Or maybe not.

Bolstered by my enlightenment that you can upgrade pip using easy_install, I proceed to spin up new virtualenvs, just to savor the feeling of having an upgraded pip at last.

(spam) chad@scheherazade:~$ pip freeze
Traceback (most recent call last):
  File "/home/chad/.virtualenvs/spam/bin/pip", line 7, in 
    from pip._internal import main
ImportError: No module named 'pip._internal'

Thankfully, as a Professional Software Engineer with Years in the Industry, I can confidently read and debug the source code of the tools I am using. So I figured out that I should change line 7 of

# -*- coding: utf-8 -*-
import re
import sys
from pip._internal import main
if __name__ == '__main__':
    sys.argv[0] = re.sub(r'(-script\.pyw|\.exe)?$', '', sys.argv[0])


from pip import main

And everything should work fine and dandy again…

…except for the pip version:

(spam) chad@scheherazade:~$ pip --version
pip 8.1.2 from /home/chad/.venvburrito/lib/python2.7/site-packages/pip-8.1.2-py2.7.egg (python 3.5)

I remember first encountering this all the way back at 2012, under, I think, 10.04 Lucid Lynx. But back then, I had a potato for a computer–the kind which weeps against Eclipse–and Ubuntu isn’t as polished as it is now. But, for crying out loud, it is 2018, I am on machines with glorious i7 processors and at least 16GB of RAM and this shit is still a thing.

At this point, Python has gone from a language whose ecosystem just works to a time waster to rival all the times I had to manually type public static void main(String[] args) just to test a damn Java behavior. I call it quits and I’m just happy I even have virtualenvs with a working pip installation.

…encountering new errors along the way, as well.

chad@scheherazade:pydagogical$ python3 -m ai.ants
Ant 0 is a touring machine.
Traceback (most recent call last):
  File "/usr/lib/python3.5/", line 253, in choice
    i = self._randbelow(len(seq))
  File "/usr/lib/python3.5/", line 230, in _randbelow
    r = getrandbits(k)          # 0 <= r < 2**k
ValueError: number of bits must be greater than zero
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
  File "/usr/lib/python3.5/", line 184, in _run_module_as_main
    "__main__", mod_spec)
  File "/usr/lib/python3.5/", line 85, in _run_code
    exec(code, run_globals)
  File "/home/chad/kode/pydagogical/ai/", line 85, in <module>
  File "/home/chad/kode/pydagogical/ai/", line 57, in solve
  File "/home/chad/kode/pydagogical/ai/", line 41, in ant_tour
    next_city = self.get_random_next(self.antroutes[antno], current)
  File "/home/chad/kode/pydagogical/ai/", line 52, in get_random_next
    next_city = random.choice(choices[:int(len(choices) / 4)])
  File "/usr/lib/python3.5/", line 255, in choice
    raise IndexError('Cannot choose from an empty sequence')
IndexError: Cannot choose from an empty sequence

Exception during an exception? Wow.

I just got locked out of my Linux machine (details here) for reasons I am still trying to determine (see the link). I tried to recover my password but, unfortunately, all my most recent Live CDs I’ve left at work. All I have with me is an old Karmic installer, incidentally, the first Ubuntu I ever tried to install (and failed miserably more due to hubris and absence of mind—but that’s another story).

So from the Live CD, I was able to mount my Linux partition fine. Note that, I guess due to EFI and/or the tutorial I was following is just outdated, a mere sudo fdisk -l did not show me all my partitions; I had to use gparted (thankfully available with Karmic). But when I try to chroot, into my mounted partition, I get an exec format error which is apparently because my Karmic LiveCD is just 32-bit while the installation I am trying to salvage is 64-bit.

Thankfully, tinkering around with how Linux stores passwords, I got to reset my main account to have no password.

  1. Edit /etc/passwd. Look for your username in the first of the colon-delimited fields. A typical line in the this file looks like bin:x:2:2:bin:/bin:/bin/sh. The ‘x’ in the second field means that the password for this account is encrypted. Remove the x so the line corresponding to your user becomes similar to bin::2:2:bin:/bin:/bin/sh.
  2. Your actual password is stored in the /etc/shadow file. Open this file and, again, edit the line corresponding to your user to just have four colons after it like so: user::::.
  3. Reboot. You should now be able to log-in passwordless. Remember to reset your password at once.

Note that to get my system to boot from LiveCD I had to tweak my BIOS yet again. I had to set my boot mode to Legacy and, after doing the steps outlined above, had to switch back to UEFI and disable Secure Boot. Man is BIOS dealing becoming traditional for me!

Yay. I just defeated Miraak, making the Dragonborn DLC the first story arc I actually finished in Skyrim. I did not really intend to make it the first, it’s just that, Miraak’s theft of the souls of the dragons I slay has started to become a pain in the neck. So I decided to finally do him in.

However, I encountered a real show stopper of a bug along the way. I found a workaround to it myself, but one with which I’m not very pleased.

I haven’t found this bug described anywhere else so if anyone else encounters the bug, here goes…

Bug behavior: Upon dismounting Sahrotaar to face Miraak, the Dragonborn is instantly killed. Note that I dismount while Miraak is making his little welcome speech.

Dragonborn glows green and dies. No explanations whatsoever. What a way to finally face an arch-nemesis, keeling over to instant death.


  1. When the game prompts you to make a landing (e.g., in PC, the text “Press E to land” is displayed), do not land. Steer Sahrotaar to the Summit of Apocrypha and use the Bend Will shout once more. This will make Sahrotaar take ground without the Dragonborn dismounting.
  2. I’m not sure if you can order Sahrotaar to interrupt Miraak’s speech and just do battle but I sat through it.
  3. When the battle starts, you will have to battle while riding Sahrotaar. You can order him to attack Miraak, but not the other two dragons. Miraak behaves normally (i.e., keeps attacking you, the Dragonborn).
  4. Miraak still kills Sahrotaar last. When Sahrotaar is killed, the Dragonborn is automatically dismounted from him. Beware that, at this moment in the fight, the Dragonborn becomes very open for Miraak’s attack. Plan your actions well after this forced dismount. (This caught me by surprise but I was able to handle it with a well-timed consumption of a healing potion or two).
  5. Continue the battle. At this point, Miraak only needs to be weakened once more before Hermaeus Mora finally ends him.



  • My biggest annoyance with my workaround is in how it cut the battle with Miraak very short. Since, in the first three rounds, I’m ordering Sahrotaar to attack Miraak, Miraak was quickly weakened. I appreciate the leg-up but I really prepared to face Miraak: items, skills, tactic research, game plan preparation, the works. It sucks that I only get a single round out of four; Sahrotaar took the first three. And even the first three rounds did not last long as Sahrotaar was dealing damage way more than what my Dragonborn could’ve dealt.
  • As I noted in (3), the fight goes on even if you have not yet dismounted Sahrotaar. I’m not sure how Skyrim implements battles while dragon riding, but will it be possible for Sahrotaar to have died before his turn to heal Miraak? Judging by the wiki bugs list for this quest, in the event that Sahrotaar dies before healing Miraak, the quest may not complete.
  • Some experiments with my workaround would be interesting, but I haven’t got the proper save points to try it out. For one, would it be okay to dismount once the battle has started (and remedy my big annoyance described above)?

For the record, here’s the kind of battle I prepared for:

In contrast, I think my whole battle with Miraak did not even go beyond five minutes.

The First Dragonborn is dead. Long live the Last Dragonborn!