Skip navigation

Tag Archives: ubuntu

Then, Again, Nothing is Perfect

I love Python. I think it is a very productive language for prototyping, pedagogy, and experimentation with a mature ecosystem to support you as you take your shit code ideas to production. However, in all my years using it, I could never get to upgrade pip, its preferred package manager, inside a virtualenv without problems. And that frustrates me.

The story starts with one of the first things you tell Python noobs before they pwn their own system and blame it all on hallowed Python: use virtualenvs. I’m not sure if this applies to Windows users (or if the concept of virtualenvs even applies to Windows users) but for *nix users, this advice makes sense since Python is often used by your system for other, more-important-than-your-side-projects, OS-y things it does. This is why Python often comes pre-packaged with the system and why you should use virtualenvs: the last thing you want is to upgrade the datetime library which your NTP daemon is using only to find out that said daemon relies on a particular quirk of that datetime library version to function properly. And so virtualenvs.

I do not intend to discuss how virtualenvs work, but suffice it to say that it does magic so that the libraries your projects need are isolated. If that whole story about NTP daemons and datetime above just flew over your head, at least appreciate how nice it is to have your dependencies isolated like that; alexandria could not see PyGame, and PyGame Objects could not see Flask, and yet I develop them on the same machine. It tingles that part of my brain which makes me put my glass/cup down at different spots on the table every time after I take a sip so that I end up with liquid circles touching each other but not overlapping. An ordered world is just so zen.

(Although I should note that I’ve long given up making PyGame work consistently with virtualenvs across the machines I use. Nowadays, I just containerize the whole thing to find almost the same peace of mind.)

Anyway, I’m beginning to digress. To proceed with the story, there comes a time when issuing pip commands inside a virtualenv results to the following warning:

(origin3) chad@scheherazade:~$ pip freeze
pbr==1.10.0
six==1.10.0
stevedore==1.18.0
virtualenv==15.0.3
virtualenv-clone==0.2.6
virtualenvwrapper==4.7.2
You are using pip version 8.1.2, however version 10.0.1 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.

And, well, who does not want free updates? Not to mention when the software involved is not just free but also open-source. And have I told you that this free and open-source software is part of the ecosystem of my favorite programming language? Up-update and away, I say!

(origin3) chad@scheherazade:~$ pip install --upgrade pip
Collecting pip
  Using cached https://files.pythonhosted.org/packages/0f/74/ecd13431bcc456ed390b44c8a6e917c1820365cbebcb6a8974d1cd045ab4/pip-10.0.1-py2.py3-none-any.whl
Installing collected packages: pip
  Found existing installation: pip 8.1.2
    Not uninstalling pip at /home/chad/.venvburrito/lib/python2.7/site-packages/pip-8.1.2-py2.7.egg, outside environment /home/chad/.virtualenvs/origin3
Successfully installed pip-8.1.2
You are using pip version 8.1.2, however version 10.0.1 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.

Err…okay, that did not really work. But not to worry! The beauty of using FOSS is that help is almost always available online. Especially on StackOverflow, a pillar of modern software engineering. Which leads us to the following words of wisdom:

pip is just a PyPI package like any other; you could use it to upgrade itself the same way you would upgrade any package:

~Cairnarvon on StackOverflow

But, but…Cairnarvon good sir, said instructions not working is what exactly set me on this quest! Then again, no one else is complaining that the accepted answer does not really work so I guess I’m the idiot here.

Thankfully, there are more words of wisdom in StackOverflow. If one option does not work, you have options. Fortunate then that this idiot can read. Since “pip is just a PyPI package”, it stands to reason that you can also upgrade it using its totally-not-sexy sibling of a package manager, easy_install.

Now, the other thing you tell Python noobs, maybe even before you tell them about virtualenvs, is to use pip over easy_install because reasons. So at this point, I am conflicted: one of the most ingrained teachings in my being is to use pip over easy_install but I have also been told to use updated software packages as much as possible. The latter urge wins out so I end up doing naughty things…

(origin3) chad@scheherazade:~$ easy_install pip
Searching for pip
Best match: pip 10.0.1
Adding pip 10.0.1 to easy-install.pth file
Installing pip3.6 script to /home/chad/.virtualenvs/origin3/bin
Installing pip3 script to /home/chad/.virtualenvs/origin3/bin
Installing pip script to /home/chad/.virtualenvs/origin3/bin

Using /home/chad/.virtualenvs/origin3/lib/python3.5/site-packages
Processing dependencies for pip
Finished processing dependencies for pip

…with good results:

(origin3) chad@scheherazade:pydagogical$ pip --version
pip 10.0.1 from /home/chad/.virtualenvs/origin3/lib/python3.5/site-packages/pip (python 3.5)

Or maybe not.

Bolstered by my enlightenment that you can upgrade pip using easy_install, I proceed to spin up new virtualenvs, just to savor the feeling of having an upgraded pip at last.

(spam) chad@scheherazade:~$ pip freeze
Traceback (most recent call last):
  File "/home/chad/.virtualenvs/spam/bin/pip", line 7, in 
    from pip._internal import main
ImportError: No module named 'pip._internal'

Thankfully, as a Professional Software Engineer with Years in the Industry, I can confidently read and debug the source code of the tools I am using. So I figured out that I should change line 7 of

1
2
3
4
5
6
7
8
9
10
11

#!/home/chad/.virtualenvs/spam/bin/python3
 
# -*- coding: utf-8 -*-
import re
import sys
 
from pip._internal import main
 
if __name__ == '__main__':
    sys.argv[0] = re.sub(r'(-script\.pyw|\.exe)?$', '', sys.argv[0])
    sys.exit(main())

to

from pip import main

And everything should work fine and dandy again…

…except for the pip version:

(spam) chad@scheherazade:~$ pip --version
pip 8.1.2 from /home/chad/.venvburrito/lib/python2.7/site-packages/pip-8.1.2-py2.7.egg (python 3.5)

I remember first encountering this all the way back at 2012, under, I think, 10.04 Lucid Lynx. But back then, I had a potato for a computer–the kind which weeps against Eclipse–and Ubuntu isn’t as polished as it is now. But, for crying out loud, it is 2018, I am on machines with glorious i7 processors and at least 16GB of RAM and this shit is still a thing.

At this point, Python has gone from a language whose ecosystem just works to a time waster to rival all the times I had to manually type public static void main(String[] args) just to test a damn Java behavior. I call it quits and I’m just happy I even have virtualenvs with a working pip installation.

Playing MP3 Files in Java

Following up from my previous post, I have decided to use Java for Museic since, ironically, it looks easier to write platform-portable code that plays MP3 files in Java than in Python. And well it is but that is not to say that the road to my goal was bump-free.

First off, the “default” way to play MP3 files in Java would be to use the JavaFX library. However, JavaFX is not at all available in Java 6 (you need to include specific JARs in your class path to use it), only half-included in Java 7 (you still need to be specific with your class path), and fully-included only in Java 8.

The obvious way to stop worrying at all about the version of Java installed in your system is to use a dependency manager, my choice being Maven. Maven has a JavaFX plugin. All you need to do is (1) fix your class path by issuing the command mvn com.zenjava:javafx-maven-plugin:2.0:fix-classpath and (2) add the following in your pom.xml

<plugin>
    <groupId>com.zenjava</groupId>
    <artifactId>javafx-maven-plugin</artifactId>
    <version>2.0</version>
    <configuration>
        <mainClass>[put your application main class here]</mainClass>
    </configuration>
</plugin>

Alas, I wish it was that easy. In my system, it was complaining about some missing JARs. It seemed to me that I was encountering the very problems I hoped to avoid by using Maven.

Then I realized that I was using OpenJDK 7 instead of Oracle JDK 7. Now, I’m aware of the distinction between OpenJDK and Oracle JDK; at Chikka, we’ve made it a point to use Oracle JDK in production. Nonetheless, for my personal projects, I’ve never experienced any significant difference of OpenJDK so I kept using it as it took less effort to install in Ubuntu. Until now.

The moral of the story is: if you are going to use JavaFX, make sure you are using Oracle JDK.

(And yes, Museic is now playing MP3 files with my desired delay, thank you very much. It’s now my practice/jam buddy!)

Playing MP3 Files Programatically

From my last post, I’m taking a little break from Chess as I turn my attention to a guitar I bought as a Christmas gift to myself. You see, I’m in the process of becoming a Guitar God, ala Jason Mraz, but that story is better reserved for my main blog no?

And well, since I can’t personally jam with Jason Mraz (or Chris Martin), I settle for MP3’s of his songs. The problem is, MP3s play abruptly without warning. There is no sufficient time between my hand pressing play and the song playing!

Jason Mraz Fanboy

And yes, I do have the CDs!

None of the MP3 players I know of is aspiring-musician-friendly as such. So, obviously, this makes it a nice candidate for a quick hack.

My requirements are simple:

  • Have a UI
  • Be able to play an MP3 file after a set countdown. And it has to be in MP3 format, hard requirement. If it is to be of any use with minimal hassles, it has to be MP3.

 

The first requirement makes it a good candidate for a web/JavaScript app. But then, as far as I know, JavaScript is not allowed access to the local filesystem for security purposes. So JavaScript (unfortunately) rules out the more-important requirement of the two I have.

The next obvious choice is Java since I’m quite used to utilizing Swing to develop desktop GUIs; in fact, I created the user interface of our thesis in Swing, from ground up. However, for a quick hack, I think Java might be overkill and there’s this Python library I’ve been wanting to play with for so long…

Kivy!

Kivy can handle my UI requirement, and rather beautifully so, if I may say. Now, searching around, there are multiple ways to play an MP3 file with Python. One of them is even a Kivy library! The other libraries I found are mp3play, musicplayer, and PyGame.

But one by one, problems emerged

  • PyGame’s mp3 support is limited.
  • The sample code in musicplayer’s PyPi page looks too complicated. (Yes, since this is just a quick project, I’m in lazy mode.)
  • mp3play only works for Windows XP.

 

And then the Kivy library. It would’ve been sweet to have a single library for all my project’s requirements. But alas…

Kivy has a SoundLoader class which automagically determines the best way to handle the given sound file’s format. I’m not good with design pattern terminology but, the way I see it, it looks like a combination of Factory and Strategy patterns. However, when I test code given in their documentation, I come across the following logs:

[INFO ] Kivy v1.8.0
[INFO ] [Logger ] Record log in /home/chad/.kivy/logs/kivy_15-01-04_6.txt
[DEBUG ] [Audio ] register SoundPygame
[INFO ] [Audio ] Providers: audio_pygame (audio_pygst, audio_sdl ignored)
[WARNING] [Audio ] Unable to find a loader for <test.mp3>

 

The only provider that loaded is PyGame’s. And, as I’ve noted above, PyGame’s mp3 support is shaky. In fact, Kivy’s code as of presstime only allows PyGame on MP3 files if it is running on Android. audio_pygst would’ve done the trick but then, as the logs indicate, it did not load.

So why did it not load? Looking at audio_pygst’s code, it imports the modules gi, pygst, and gst, in that order, respectively surrounded in try-catch statements should there be problems on the import.

I try to load them manually on Python’s shell and get the following result:

chad@galadriel:kivy$ python
Python 2.7.3 (default, Feb 27 2014, 19:58:35) 
[GCC 4.6.3] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import gi
>>> import pygst
>>> import gst
/usr/lib/python2.7/dist-packages/gobject/constants.py:24: Warning: g_boxed_type_register_static: assertion `g_type_from_name (name) == 0' failed
  import gobject._gobject
>>>

 

Looking around, this seems to be an issue with gst 0.10.x. The obvious solution would be to upgrade. Unfortunately, gst updates for Ubuntu 12.04LTS seem to have stopped at 0.10.

So there. Much ado achieving nothing, for a quick hack. Maybe, I’ll look into using Java for this project, moving forward. Stay tuned!

Coldplay Fanboy

Change Your Linux Password Using an Older Live CD

I just got locked out of my Linux machine (details here) for reasons I am still trying to determine (see the link). I tried to recover my password but, unfortunately, all my most recent Live CDs I’ve left at work. All I have with me is an old Karmic installer, incidentally, the first Ubuntu I ever tried to install (and failed miserably more due to hubris and absence of mind—but that’s another story).

So from the Live CD, I was able to mount my Linux partition fine. Note that, I guess due to EFI and/or the tutorial I was following is just outdated, a mere sudo fdisk -l did not show me all my partitions; I had to use gparted (thankfully available with Karmic). But when I try to chroot, into my mounted partition, I get an exec format error which is apparently because my Karmic LiveCD is just 32-bit while the installation I am trying to salvage is 64-bit.

Thankfully, tinkering around with how Linux stores passwords, I got to reset my main account to have no password.

  1. Edit /etc/passwd. Look for your username in the first of the colon-delimited fields. A typical line in the this file looks like bin:x:2:2:bin:/bin:/bin/sh. The ‘x’ in the second field means that the password for this account is encrypted. Remove the x so the line corresponding to your user becomes similar to bin::2:2:bin:/bin:/bin/sh.
  2. Your actual password is stored in the /etc/shadow file. Open this file and, again, edit the line corresponding to your user to just have four colons after it like so: user::::.
  3. Reboot. You should now be able to log-in passwordless. Remember to reset your password at once.

Note that to get my system to boot from LiveCD I had to tweak my BIOS yet again. I had to set my boot mode to Legacy and, after doing the steps outlined above, had to switch back to UEFI and disable Secure Boot. Man is BIOS dealing becoming traditional for me!

Ubuntu WiFi Problems

Ever since I got my new laptop and upgraded to Ubuntu 12.04, I noticed that my WiFi has been very unsteady to the point that for anything greater than 100MB, my most reliable download method would be via torrent, unless the server I’m downloading from supports resuming downloads, which is never. That’s a shame since not everything > 100MB can be found in P2P networks. I also noticed that Ubuntu 12.04 oftentimes took ages to connect to our WiFi network upon start-up. This isn’t the case with 10.04 and Windows 8. Unfortunately, the last time I felt this frustrated with WiFi connection upon start-up was with Windows Vista.

At first I thought it is an issue with our router which is around 3-4 years old—a suspicion I got pretty confident with upon checking my ping and traceroute logs. My pings averaged at around hundreds of milliseconds, sometimes even up to 800ms, while the normal was only at around 50ms to 60ms. Traceroute, on the other hand, times out intermittently and, noticeably, on the first hop towards our router (192.168.0.1).

I got a replacement for our router around a month ago and, to my disappointment, the problem persisted. My ping and traceroute returned to normal but the WiFi remained choppy while connected and initial connection upon start-up still took ages. I’ve been searching for a solution for this problem but every solution I come across seem to be very hardware specific (check those network cards, geeks) and involved building some low-level modules. For what it’s worth, it seems that this problem boils down to a conflict among the wireless drivers that come with 12.04.

Then somewhere, in my searches, I encountered a post in Ubuntu Forums which looked like a general solution to the problem I am having. This, as the command suggests, installs a backport of wireless drivers into your machine:

sudo apt-get install linux-backports-modules-cw-3.3-precise-generic

Now, my Ubuntu 12.04 instantly connects to our WiFi upon start-up, just like my old 10.04 installation.

The WiFi still cuts though, something which I’ve pinpointed to be a fault of the router (or, maybe, how I’m positioned with respect to the router): I noticed that my XPeria Z, stationed on the same desk, also gets disconnected. For what it’s worth, reestablishing connection is now faster since I backported my drivers.