Skip navigation

Tag Archives: python

Coming off from my last post about updating pip inside a virtualenv, I have, finally, realized what I’ve been doing wrong all this time: I use this nifty script called virtualenv burrito to get a virtualenv + virtualenvwrapper set-up.

I’ve been writing an install script for a new project I am working on. Being at such an early phase, the script is scrappy and will only probably work for machines configured exactly as mine. Not that I make a lot of customizations, but still it relies on the existence of virtualenvwrapper too much.

The thing is I could never get my install script to run properly since pip is broken in my virtualenvs. So just yesterday, I finally decided to devote some time to digging into why precisely this is happening.

Long story short, after a few hours digging around the Vogon poetry of bash scripting, I realized that I can’t find separate virtualenv and virtualenvwrapper installs in my system…because I used virtualenv burrito to install them.

So I deleted the .venvburrito directory in my home and rerun virtualenv burrito. And…tadddaaaahhh!

chad@scheherazade:~$ mkvirtualenv fresh
New python executable in /home/chad/.virtualenvs/fresh/bin/python
Installing setuptools, pip, wheel...done.
virtualenvwrapper.user_scripts creating /home/chad/.virtualenvs/fresh/bin/predeactivate
virtualenvwrapper.user_scripts creating /home/chad/.virtualenvs/fresh/bin/postdeactivate
virtualenvwrapper.user_scripts creating /home/chad/.virtualenvs/fresh/bin/preactivate
virtualenvwrapper.user_scripts creating /home/chad/.virtualenvs/fresh/bin/postactivate
virtualenvwrapper.user_scripts creating /home/chad/.virtualenvs/fresh/bin/get_env_details
(fresh) chad@scheherazade:~$ 
(fresh) chad@scheherazade:~$ pip -V
pip 10.0.1 from /home/chad/.venvburrito/lib/python2.7/site-packages/pip-10.0.1-py2.7.egg/pip (python 2.7

What this does not address, however, are my old virtualenvs which are stuck on an outdated pip. But I guess it should be trivial to just delete those virtualenvs and create a new one, this time with an updated pip.

There’s this thing in life, where you have assumptions you neglect to state because you thought (assume) them to be inconsequential, then they turn out to be of great importance. This is all the more true when you work with computers.

I love Python. I think it is a very productive language for prototyping, pedagogy, and experimentation with a mature ecosystem to support you as you take your shit code ideas to production. However, in all my years using it, I could never get to upgrade pip, its preferred package manager, inside a virtualenv without problems. And that frustrates me.

The story starts with one of the first things you tell Python noobs before they pwn their own system and blame it all on hallowed Python: use virtualenvs. I’m not sure if this applies to Windows users (or if the concept of virtualenvs even applies to Windows users) but for *nix users, this advice makes sense since Python is often used by your system for other, more-important-than-your-side-projects, OS-y things it does. This is why Python often comes pre-packaged with the system and why you should use virtualenvs: the last thing you want is to upgrade the datetime library which your NTP daemon is using only to find out that said daemon relies on a particular quirk of that datetime library version to function properly. And so virtualenvs.

I do not intend to discuss how virtualenvs work, but suffice it to say that it does magic so that the libraries your projects need are isolated. If that whole story about NTP daemons and datetime above just flew over your head, at least appreciate how nice it is to have your dependencies isolated like that; alexandria could not see PyGame, and PyGame Objects could not see Flask, and yet I develop them on the same machine. It tingles that part of my brain which makes me put my glass/cup down at different spots on the table every time after I take a sip so that I end up with liquid circles touching each other but not overlapping. An ordered world is just so zen.

(Although I should note that I’ve long given up making PyGame work consistently with virtualenvs across the machines I use. Nowadays, I just containerize the whole thing to find almost the same peace of mind.)

Anyway, I’m beginning to digress. To proceed with the story, there comes a time when issuing pip commands inside a virtualenv results to the following warning:

(origin3) chad@scheherazade:~$ pip freeze
pbr==1.10.0
six==1.10.0
stevedore==1.18.0
virtualenv==15.0.3
virtualenv-clone==0.2.6
virtualenvwrapper==4.7.2
You are using pip version 8.1.2, however version 10.0.1 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.

And, well, who does not want free updates? Not to mention when the software involved is not just free but also open-source. And have I told you that this free and open-source software is part of the ecosystem of my favorite programming language? Up-update and away, I say!

(origin3) chad@scheherazade:~$ pip install --upgrade pip
Collecting pip
  Using cached https://files.pythonhosted.org/packages/0f/74/ecd13431bcc456ed390b44c8a6e917c1820365cbebcb6a8974d1cd045ab4/pip-10.0.1-py2.py3-none-any.whl
Installing collected packages: pip
  Found existing installation: pip 8.1.2
    Not uninstalling pip at /home/chad/.venvburrito/lib/python2.7/site-packages/pip-8.1.2-py2.7.egg, outside environment /home/chad/.virtualenvs/origin3
Successfully installed pip-8.1.2
You are using pip version 8.1.2, however version 10.0.1 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.

Err…okay, that did not really work. But not to worry! The beauty of using FOSS is that help is almost always available online. Especially on StackOverflow, a pillar of modern software engineering. Which leads us to the following words of wisdom:

pip is just a PyPI package like any other; you could use it to upgrade itself the same way you would upgrade any package:

~Cairnarvon on StackOverflow

But, but…Cairnarvon good sir, said instructions not working is what exactly set me on this quest! Then again, no one else is complaining that the accepted answer does not really work so I guess I’m the idiot here.

Thankfully, there are more words of wisdom in StackOverflow. If one option does not work, you have options. Fortunate then that this idiot can read. Since “pip is just a PyPI package”, it stands to reason that you can also upgrade it using its totally-not-sexy sibling of a package manager, easy_install.

Now, the other thing you tell Python noobs, maybe even before you tell them about virtualenvs, is to use pip over easy_install because reasons. So at this point, I am conflicted: one of the most ingrained teachings in my being is to use pip over easy_install but I have also been told to use updated software packages as much as possible. The latter urge wins out so I end up doing naughty things…

(origin3) chad@scheherazade:~$ easy_install pip
Searching for pip
Best match: pip 10.0.1
Adding pip 10.0.1 to easy-install.pth file
Installing pip3.6 script to /home/chad/.virtualenvs/origin3/bin
Installing pip3 script to /home/chad/.virtualenvs/origin3/bin
Installing pip script to /home/chad/.virtualenvs/origin3/bin

Using /home/chad/.virtualenvs/origin3/lib/python3.5/site-packages
Processing dependencies for pip
Finished processing dependencies for pip

…with good results:

(origin3) chad@scheherazade:pydagogical$ pip --version
pip 10.0.1 from /home/chad/.virtualenvs/origin3/lib/python3.5/site-packages/pip (python 3.5)

Or maybe not.

Bolstered by my enlightenment that you can upgrade pip using easy_install, I proceed to spin up new virtualenvs, just to savor the feeling of having an upgraded pip at last.

(spam) chad@scheherazade:~$ pip freeze
Traceback (most recent call last):
  File "/home/chad/.virtualenvs/spam/bin/pip", line 7, in 
    from pip._internal import main
ImportError: No module named 'pip._internal'

Thankfully, as a Professional Software Engineer with Years in the Industry, I can confidently read and debug the source code of the tools I am using. So I figured out that I should change line 7 of

1
2
3
4
5
6
7
8
9
10
11
#!/home/chad/.virtualenvs/spam/bin/python3
 
# -*- coding: utf-8 -*-
import re
import sys
 
from pip._internal import main
 
if __name__ == '__main__':
    sys.argv[0] = re.sub(r'(-script\.pyw|\.exe)?$', '', sys.argv[0])
    sys.exit(main())

to

from pip import main

And everything should work fine and dandy again…

…except for the pip version:

(spam) chad@scheherazade:~$ pip --version
pip 8.1.2 from /home/chad/.venvburrito/lib/python2.7/site-packages/pip-8.1.2-py2.7.egg (python 3.5)

I remember first encountering this all the way back at 2012, under, I think, 10.04 Lucid Lynx. But back then, I had a potato for a computer–the kind which weeps against Eclipse–and Ubuntu isn’t as polished as it is now. But, for crying out loud, it is 2018, I am on machines with glorious i7 processors and at least 16GB of RAM and this shit is still a thing.

At this point, Python has gone from a language whose ecosystem just works to a time waster to rival all the times I had to manually type public static void main(String[] args) just to test a damn Java behavior. I call it quits and I’m just happy I even have virtualenvs with a working pip installation.

A full GitHub year

I technically hit the year mark on January 6 but I wanted a full perfect rectangle of green so I pushed for a week more and this is what I got.

I think it’s time to erm…cash out on the things I learned doing this thing. Or spend more time thinking and less of coding; but I think, the habit can be quite hard to break.

A few more words before I go, the year mostly revolved around two projects, interestingly both a throwback from when I was just fresh out of university.

alexandria is actually a Python “port” of my earliest project on GitHub. Scare quotes on the word “port” because, by now, it hardly is a port. I wonder if the old PHP code will still run if I try to set it up today. The Python port itself, however, was started in 2015.

PyGame-Objects is my earliest attempts at seriously Python-ing. I created Snake this year which is, yeah, not very impressive but it really made me happy to try to write it as cleanly as I could, modifying the framework (which I built too, which is what PyGame Objects is all about) when needed, and maintaining backwards-compatibility with the previous games. I think some time ago (in 2013? But that feels like a lifetime now) I tried to rename it to something cooler but I stuck with the humbler name for now, and for the foreseeable future.

I cannot contain my excitement when the debug mode for PyGame Objects finally worked. Oh what joy! It felt legit even if there is much work to be done.

Pardon my seeming lack of modesty but my great takeaway here is that I write code pretty clean. They say a good rule-of-the-thumb of code cleanliness and documentation is if you can understand code you last touched three months ago. Well I last touched most of PyGame-Objects four years ago!

But I think, I overshot my intended quota of words. Goodbye now. Gotta play The Witcher 3: Wild Hunt.

From my last post, I’m taking a little break from Chess as I turn my attention to a guitar I bought as a Christmas gift to myself. You see, I’m in the process of becoming a Guitar God, ala Jason Mraz, but that story is better reserved for my main blog no?

And well, since I can’t personally jam with Jason Mraz (or Chris Martin), I settle for MP3’s of his songs. The problem is, MP3s play abruptly without warning. There is no sufficient time between my hand pressing play and the song playing!

Jason Mraz Fanboy

And yes, I do have the CDs!

None of the MP3 players I know of is aspiring-musician-friendly as such. So, obviously, this makes it a nice candidate for a quick hack.

My requirements are simple:

  • Have a UI
  • Be able to play an MP3 file after a set countdown. And it has to be in MP3 format, hard requirement. If it is to be of any use with minimal hassles, it has to be MP3.

 

The first requirement makes it a good candidate for a web/JavaScript app. But then, as far as I know, JavaScript is not allowed access to the local filesystem for security purposes. So JavaScript (unfortunately) rules out the more-important requirement of the two I have.

The next obvious choice is Java since I’m quite used to utilizing Swing to develop desktop GUIs; in fact, I created the user interface of our thesis in Swing, from ground up. However, for a quick hack, I think Java might be overkill and there’s this Python library I’ve been wanting to play with for so long…

Kivy!

Kivy can handle my UI requirement, and rather beautifully so, if I may say. Now, searching around, there are multiple ways to play an MP3 file with Python. One of them is even a Kivy library! The other libraries I found are mp3play, musicplayer, and PyGame.

But one by one, problems emerged

  • PyGame’s mp3 support is limited.
  • The sample code in musicplayer’s PyPi page looks too complicated. (Yes, since this is just a quick project, I’m in lazy mode.)
  • mp3play only works for Windows XP.

 

And then the Kivy library. It would’ve been sweet to have a single library for all my project’s requirements. But alas…

Kivy has a SoundLoader class which automagically determines the best way to handle the given sound file’s format. I’m not good with design pattern terminology but, the way I see it, it looks like a combination of Factory and Strategy patterns. However, when I test code given in their documentation, I come across the following logs:

[INFO ] Kivy v1.8.0
[INFO ] [Logger ] Record log in /home/chad/.kivy/logs/kivy_15-01-04_6.txt
[DEBUG ] [Audio ] register SoundPygame
[INFO ] [Audio ] Providers: audio_pygame (audio_pygst, audio_sdl ignored)
[WARNING] [Audio ] Unable to find a loader for <test.mp3>

 

The only provider that loaded is PyGame’s. And, as I’ve noted above, PyGame’s mp3 support is shaky. In fact, Kivy’s code as of presstime only allows PyGame on MP3 files if it is running on Android. audio_pygst would’ve done the trick but then, as the logs indicate, it did not load.

So why did it not load? Looking at audio_pygst’s code, it imports the modules gi, pygst, and gst, in that order, respectively surrounded in try-catch statements should there be problems on the import.

I try to load them manually on Python’s shell and get the following result:

chad@galadriel:kivy$ python
Python 2.7.3 (default, Feb 27 2014, 19:58:35) 
[GCC 4.6.3] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import gi
>>> import pygst
>>> import gst
/usr/lib/python2.7/dist-packages/gobject/constants.py:24: Warning: g_boxed_type_register_static: assertion `g_type_from_name (name) == 0' failed
  import gobject._gobject
>>>

 

Looking around, this seems to be an issue with gst 0.10.x. The obvious solution would be to upgrade. Unfortunately, gst updates for Ubuntu 12.04LTS seem to have stopped at 0.10.

So there. Much ado achieving nothing, for a quick hack. Maybe, I’ll look into using Java for this project, moving forward. Stay tuned!

Coldplay Fanboy

Okay. First off, I’m aware that I’ve lagged behind my New Year’s Resolution. But I’ve only lagged behind the blogging part: I’m still exploring new stuff monthly. Guess I’ll talk about them in brief for now and, maybe, expound some other time. ^_^

Just to clear some smoke, last time, I told you that I may be in for a volunteer project. Well, so I’ve been and yes, I did use Django pretty extensively with it. The project is for the up-and-coming Python Philippines. I’m the one behind the website and it’s running on Django CMS. Go visit it. We don’t have content yet but, well, you can look at the pretty theme. Hehe.

And just last Saturday, I talked about Django CMS at Python Philippines’ monthly meet-up, to encourage contributors. I feel that my talk went well and I hope to put up a transcript of it in here, to make up for my lack of blogging. Soon!

 

I’ve also been trying to learn some things from the ground up in Python. I’ve done sockets last time and this time around, I tried doing threads. Straight from the box, Python does not really support threads. You’d need libraries like gevent to actually get some concurrency done in peace. But still, unlike PHP, if you just want to switch among jobs, Python has something for you.

I’ve experimented with Python’s threading library. It’s a curious thing, coming from Java’s threads. I got so curious with it I’m even trying to learn statistics to validate my experiments. Because, you know, I did all my samples in powers/multiples of ten.

 

Lastly, as a form of humbrag1, I got myself an Xperia Z, for around a month now2. I must say, it’s damn gorgeous! I never got people who complain that Samsung uses cheap plastic to house impressive hardware but with the Xperia Z’s all-glass build, I get it now. It’s all about the user experience, people. User. Experience.

The hardware is damn powerful; Iron Man 3 plays very smoothly, save for occasional lags. The camera isn’t shabby either; for quick shots, it can do in place of my bulky SLR. Being water proof, dust proof, and shock proof sealed the deal for me.

At last, at last, I can play Temple Run 2 even when it’s raining.

  1. Okay, really bragging mostly []
  2. Those who know the camera I am using can now scream “Fanboy!” []