Skip navigation

Category Archives: Thougts on Technical Things

I love Python. I think it is a very productive language for prototyping, pedagogy, and experimentation with a mature ecosystem to support you as you take your shit code ideas to production. However, in all my years using it, I could never get to upgrade pip, its preferred package manager, inside a virtualenv without problems. And that frustrates me.

The story starts with one of the first things you tell Python noobs before they pwn their own system and blame it all on hallowed Python: use virtualenvs. I’m not sure if this applies to Windows users (or if the concept of virtualenvs even applies to Windows users) but for *nix users, this advice makes sense since Python is often used by your system for other, more-important-than-your-side-projects, OS-y things it does. This is why Python often comes pre-packaged with the system and why you should use virtualenvs: the last thing you want is to upgrade the datetime library which your NTP daemon is using only to find out that said daemon relies on a particular quirk of that datetime library version to function properly. And so virtualenvs.

I do not intend to discuss how virtualenvs work, but suffice it to say that it does magic so that the libraries your projects need are isolated. If that whole story about NTP daemons and datetime above just flew over your head, at least appreciate how nice it is to have your dependencies isolated like that; alexandria could not see PyGame, and PyGame Objects could not see Flask, and yet I develop them on the same machine. It tingles that part of my brain which makes me put my glass/cup down at different spots on the table every time after I take a sip so that I end up with liquid circles touching each other but not overlapping. An ordered world is just so zen.

(Although I should note that I’ve long given up making PyGame work consistently with virtualenvs across the machines I use. Nowadays, I just containerize the whole thing to find almost the same peace of mind.)

Anyway, I’m beginning to digress. To proceed with the story, there comes a time when issuing pip commands inside a virtualenv results to the following warning:

(origin3) chad@scheherazade:~$ pip freeze
pbr==1.10.0
six==1.10.0
stevedore==1.18.0
virtualenv==15.0.3
virtualenv-clone==0.2.6
virtualenvwrapper==4.7.2
You are using pip version 8.1.2, however version 10.0.1 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.

And, well, who does not want free updates? Not to mention when the software involved is not just free but also open-source. And have I told you that this free and open-source software is part of the ecosystem of my favorite programming language? Up-update and away, I say!

(origin3) chad@scheherazade:~$ pip install --upgrade pip
Collecting pip
  Using cached https://files.pythonhosted.org/packages/0f/74/ecd13431bcc456ed390b44c8a6e917c1820365cbebcb6a8974d1cd045ab4/pip-10.0.1-py2.py3-none-any.whl
Installing collected packages: pip
  Found existing installation: pip 8.1.2
    Not uninstalling pip at /home/chad/.venvburrito/lib/python2.7/site-packages/pip-8.1.2-py2.7.egg, outside environment /home/chad/.virtualenvs/origin3
Successfully installed pip-8.1.2
You are using pip version 8.1.2, however version 10.0.1 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.

Err…okay, that did not really work. But not to worry! The beauty of using FOSS is that help is almost always available online. Especially on StackOverflow, a pillar of modern software engineering. Which leads us to the following words of wisdom:

pip is just a PyPI package like any other; you could use it to upgrade itself the same way you would upgrade any package:

~Cairnarvon on StackOverflow

But, but…Cairnarvon good sir, said instructions not working is what exactly set me on this quest! Then again, no one else is complaining that the accepted answer does not really work so I guess I’m the idiot here.

Thankfully, there are more words of wisdom in StackOverflow. If one option does not work, you have options. Fortunate then that this idiot can read. Since “pip is just a PyPI package”, it stands to reason that you can also upgrade it using its totally-not-sexy sibling of a package manager, easy_install.

Now, the other thing you tell Python noobs, maybe even before you tell them about virtualenvs, is to use pip over easy_install because reasons. So at this point, I am conflicted: one of the most ingrained teachings in my being is to use pip over easy_install but I have also been told to use updated software packages as much as possible. The latter urge wins out so I end up doing naughty things…

(origin3) chad@scheherazade:~$ easy_install pip
Searching for pip
Best match: pip 10.0.1
Adding pip 10.0.1 to easy-install.pth file
Installing pip3.6 script to /home/chad/.virtualenvs/origin3/bin
Installing pip3 script to /home/chad/.virtualenvs/origin3/bin
Installing pip script to /home/chad/.virtualenvs/origin3/bin

Using /home/chad/.virtualenvs/origin3/lib/python3.5/site-packages
Processing dependencies for pip
Finished processing dependencies for pip

…with good results:

(origin3) chad@scheherazade:pydagogical$ pip --version
pip 10.0.1 from /home/chad/.virtualenvs/origin3/lib/python3.5/site-packages/pip (python 3.5)

Or maybe not.

Bolstered by my enlightenment that you can upgrade pip using easy_install, I proceed to spin up new virtualenvs, just to savor the feeling of having an upgraded pip at last.

(spam) chad@scheherazade:~$ pip freeze
Traceback (most recent call last):
  File "/home/chad/.virtualenvs/spam/bin/pip", line 7, in 
    from pip._internal import main
ImportError: No module named 'pip._internal'

Thankfully, as a Professional Software Engineer with Years in the Industry, I can confidently read and debug the source code of the tools I am using. So I figured out that I should change line 7 of

1
2
3
4
5
6
7
8
9
10
11
#!/home/chad/.virtualenvs/spam/bin/python3
 
# -*- coding: utf-8 -*-
import re
import sys
 
from pip._internal import main
 
if __name__ == '__main__':
    sys.argv[0] = re.sub(r'(-script\.pyw|\.exe)?$', '', sys.argv[0])
    sys.exit(main())

to

from pip import main

And everything should work fine and dandy again…

…except for the pip version:

(spam) chad@scheherazade:~$ pip --version
pip 8.1.2 from /home/chad/.venvburrito/lib/python2.7/site-packages/pip-8.1.2-py2.7.egg (python 3.5)

I remember first encountering this all the way back at 2012, under, I think, 10.04 Lucid Lynx. But back then, I had a potato for a computer–the kind which weeps against Eclipse–and Ubuntu isn’t as polished as it is now. But, for crying out loud, it is 2018, I am on machines with glorious i7 processors and at least 16GB of RAM and this shit is still a thing.

At this point, Python has gone from a language whose ecosystem just works to a time waster to rival all the times I had to manually type public static void main(String[] args) just to test a damn Java behavior. I call it quits and I’m just happy I even have virtualenvs with a working pip installation.

Back in the day, I was really eager to work with anything you can slap an acronym on. Practice on MVCs, JDBCs, and APIs was sorely lacking in Computer Science at any level, I’d wager, but even more so in undergrad. Heck, How to Design Programs even tiered Scheme such that some library calls and language features are not available for the first several chapters. I felt that this was such a gaping hole in my employability since geeks online develop such strong feelings when an API version is deprecated.

Fortunately, my worries on employability proved baseless. No sooner than my internship did I start working on Android and its API. Then I was into a whirlwind of proprietary MVC and then some networking alphabet soup (hello, TCP, UDP, CIMD2, EMI, SMS, AT codes, ZMQ…whew). And then, in the past few months, some more API here and there. All of which has lead me to the conclusion that documentation—often overlooked, often scoffed at—is incredibly difficult to write, much less understand.

And what would make documentation “good” to begin with? In my opinion, good documentation would be

  1. Friendly to newbies with considerable skill. I add “with considerable skill” because I would not expect any reasonable documentation to start by teaching readers how to program. For a web API, let’s start with a baseline of knowing how to issue GET and POST requests, and maybe parsing JSON. And,
  2. Has a fair amount of short, self-contained, correct examples that anyone on a tight deadline can hack together something reasonably production-ready and that anyone with more time and experience can string together for something fancier. Copy-pastable examples is a huge, huge plus to this point.

(I was sorely tempted to add a sufficient, but not necessary, condition of readers being able to work without referring to StackOverflow but decided against it. Is it really reasonable to expect developers to not refer to StackOverflow, or that they can anticipate a good deal of the unique conditions in which their readers may put their libraries/services/API to use?)

I can name a handful of documents which I think will pass those two criteria. Among them,

  • Android’s documentation.
  • Most, but definitely not all, Python libraries out there.
  • Mozilla Developer Network JS and CSS is something I will find hard to live without.
  • It’s not cool to say this but the Java API docs is definitely above average.
    • And I remember that Sun’s Java tutorial—back when Sun owned Java—is a good jump-off point to the language.
  • Even less cooler to call, is CodeIgniter’s. Anyone remembers CodeIgniter? I’d have to begrudgingly give it to PHP too. Say what you want about PHP as a language but they had usable documentation back when I was actually using it.

I’m trying to come up with a list of those who don’t but I don’t want to name names, especially when documentation is so easy to change1 but, thinking about it, I think I’ve discerned a pattern: they are all, more or less, “corporate data” APIs. And cool companies are not exempt from exposing a “corporate data” API or two.

Common sins of APIs are:

  1. Being outdated. Though I’m inclined to think that frustrations which I’d bin under this label could be due to my slack as well. See I usually just Google-search whatever it is I want to do and work from what seems to be the easiest example to follow from the first page of results. Sometimes, this is StackOverflow so that lets the API writers off the hook. Sometimes, it is really the official documentation. You set things up, copy-paste the code, hit run, see things work through the logs building to that moment of anticipation when the next log should show you the data you are trying to fetch…only for your code to throw an exception. Apparently, in between the time the docs were written and now, they changed the return format of their API without it getting mentioned anywhere.
  2. Not indicating the version for which the docs were written. Slightly related to (1), I stand guilty of this too. Having mentioned my workflow above, when things don’t work, it’s become my habit to check the version for which the docs apply before going to StackOverflow.
  3. Returning unclear error messages. This is a problem that pervades programming in general; at the bare metal, things just either work or not so we created exceptions to distinguish one kind of not-working from another. But even that isn’t perfectly helpful. What do you do when the interpreter tells you it ran into a null pointer in a line of code that has several objects?2 Taken to APIs, sometimes all they tell you is that they ran into a permissions error. But what kind of permission was missing? Was it my API key? Was it permission from the user for our app to access their data? Maybe, permissions error on your OS where your endpoint tried to run a script which not even root has permission to run? Hah, which brings me to the special number in this list which is…
  4. Unnecessarily leaking your implementation to your users. This is special because there is only one (and I hope there will only be one) API I’ve worked with so far which did this. They had other fuck-ups but this is the one which made me question how exactly did my life’s choices lead me to working with them. I know we have a concept of leaky abstractions but asking your API clients to wait for 30s to act on an acknowledgement they received is just sheer incompetence, not an abstraction leaking. Do you even know how to abstract, bro?

Thankfully, if you are working with languages like Python or Javascript, introspection is dead easy which kinda works around the first two. My workarounds for those that fall into #3 has ranged from the obscure (using tcpdump on the connection and debugging on Wireshark) to the fairly standard (excessive logging and StackOverflow searching). As for four…well, thankfully, I was already serving my final 30-day notice when this detail leaked.

To be fair, I’ve had my share of designing a web API for work. I’d admit that there was a phase where our documentation wouldn’t pass my standard3 but when I left I think we were in a state that is, at least, workable, if not passing both my criteria. The title of my post could also be unfair to devs since sufficiently large companies would usually have technical writers who churn out the docs. Having technical writers, however, does not mean that devs have minimal influence on the outcome of the documentation. Based on experience, a technical writer’s job can range from just formatting/proofreading whatever the devs wrote to actually understanding the system they are documenting and actually writing most, if not all, of the documentation.

So, should devs write their own documentation? Why not. The Python libraries that pass my criteria are open-source where, I presume, devs write their own docs. The only problem—at least for most devs—with what I am espousing is that they’d actually have to learn how to write and writing effectively, technical or otherwise, requires empathy. Alas, this would be another blog post on its own.

All that said, I’m at that point in my career when, if asked to work on something you can slap an acronym on, I begin wondering when would Mangastream come around to translating and releasing their Hajime no Ippo backlog. Because best practices, bro, nothing fancy. Best. Practices.

  1. One of the things that came to my mind to call out is Facebook’s docs for setting up a lead gen endpoint. When I was working on it, all they had is a screencast walkthrough of setting up one but then months later, when I had to debug something in my implementation, I found out that they have transcribed the screencast, to my relief. The screencast, being unsearchable to begin with, is definitely not friendly to anyone with a tight deadline (I should know). Whether it is newbie-friendly or not would depend on the newbie’s preference of video over text. []
  2. You reformat your code so that there is only one object per line, heh. []
  3. At the time, the documentation wasn’t solely to blame, I think. The shape of the API then was quite convoluted for a web API that my baseline of knowing GET/POST/JSON would not be enough to work with it. []

My laptop turned one-year old last Monday. What could be called remarkable for this gadget anniversary is the fact that I had legitimate cause to claim warranty in the allotted one-year period. It’s not a show-stopper defect that required servicing. It’s something minor though it’s one of those minor things that tend to be really annoying.

One day, the down arrow-key button just stopped working (or started over working depending on how you view the symptoms).

That day I noticed that Windows 8 automagically loaded even if I missed the 10-second period where GRUB asks you for a confirmation on the OS you want to load. I did not fuss too much about it since I was about to play Skyrim, and was going to boot into Windows 8 anyway. However, later that day, when I wanted to code and so had to boot into Linux, I noticed the automagic and started wondering.

In Linux, I noticed that the browser randomly scrolls down and scrolling up with anything (mouse wheel, scroll bar drag, page-up, up arrow key) had no effect as it just drags back down. Then I started switching workspaces via the Ctrl+Alt+Down key combo which just didn’t work.

This is where I started experimenting, starting with GRUB. With no further delay, I present you, the symptoms:

  • At GRUB, I noticed that the selection cursor automatically goes down to Windows 8 (the bottom-most option). From there, while in the 10-second grace period, you can still go up (by pressing `up`) but not down.
  • Inside Linux, I noticed that Ctrl+Alt+Left does not work as well.
  • Inside Windows, while in Notepad, the down button really does not work. All the other direction keys seem to be fine though.
  • Inside a terminal session in Linux, pressing `down` does not make your terminal command history go forward.

As I said, not really a show stopper. At vim, my text editor of choice, I’ve grown accustomed to using the h, jkl keys to navigate around my document; for everything else, I have a mouse with a scroll wheel. At bash, I seldom go through my command history via the up and down buttons anyway; I mostly use Ctrl+R (reverse search). As for workspace switching, I can still get to any workspace as long as up and right are still working and wrap-around behavior is enabled. But this defect just bugs my workflow, not to mention the annoying scroll down browser behavior told earlier.

As when this happened, the machine is still under warranty, I thought it prudent to claim it. Everything’s fine, save that I went to this year’s GDG Philippines DevFest without a laptop. I had to leave it with Acer for three weeks as they ordered a new keyboard. Those three weeks, I managed to sleep on time and devote time to my other hobbies.

I wonder what could’ve caused it though.

Now, I’m having another problem, still minor but perhaps less annoying. For some reason, USB mice are working very erratically with my laptop. At first I thought that the one I’m using just failed. I’ve been using it for less than over a year and it’s not any recognizable brand but one that has sentimental value; I wouldn’t want to let it go too easily.  But I just bought an A4 D530FX, saw that it works well with my computer at work but not with my laptop. I’m not sure if I should feel happy or sad that I seem to have established that my no-namer, sentimentally-important mouse is still fine1 but my laptop may be in for another maintenance check.

(In retrospect, I could’ve saved myself a few hundred pesos by plugging my sentimentally-important mouse to another computer—like the laptop my sister is using, or the old Windows XP desktop which we only keep for printing, or the beast I use at work—and check if its working fine there. Oh well, back-ups can still be considered an investment no?)

It’s not a big problem as the touchpad is still working fine. What suffers is my Skyrim. Damn. :(

With that all said, I’m still pretty pleased with my laptop. Though I, as part of being a “wise consumer”, do check out reviews of products before buying them (especially when I buy uber-new products, like what I did with my XPeria Z), I do not place that much value in them. It seems to me that most reviews today come from people who have just used the product concerned for a test drive, just in time to write a blog post about it (and get their paycheck). Well, I’m buying things not to use them for a test drive but for a pretty long time. I’d actually like to see my purchases as investments.

After a year, I think I’m in a position to make a short review of my laptop no? Quoting from this blog’s “About” page as of this writing:

Unit Model: Acer Aspire V3
Processor: Intel Core i5 (2.5GHz) with Turbo Boost up to 3.1GHz.
RAM: 4GB DDR3
GPU: NVIDIA GeForce GT 630M with 2GB dedicated VRAM
Operating Systems: Linux Ubuntu 12.04 LTS (Precise Pangolin) and Windows 8

So well, the pros: I think it looks really sleek. It’s no MacBook, but I find the design really aesthetically pleasing. It is not as light as I would’ve wanted (when I bought this, it was due to an emergency; it was still quite out of spending schedule) but I’ve no issues with weight since, unlike when I was in undergrad, my laptop mostly sits on my desk at home now. Plus points for having the order of the Ctrl, Fn, and Super2 keys in the same order as my previous one.

The specs may not be top-notch but it is sufficient for games (SkyrimCiv V, KingsRoad, anyone?) and the side projects I’m doing. I find that Windows 8 really plays nice with it, though it is not a touchscreen model. Though I think it must be said that I’m way more pleased with Windows 8 than your average Lifehacker commenter.

And yes, there are cons: one of my very first issues with it is that it doesn’t have any indicator if I’ve my Caps Lock on. It may not be a big issue for most users but it does bug me: imagine getting your web passwords wrong only to find out that you’ve accidentally hit Caps Lock on. There’s also this weird bug it has on Google Chrome where GC suddenly hangs for no apparent reason. It happens for both Windows and Linux but Linux does not seem to be able to recover. I’ve checked the logs, reported, and Googled about this bug but I don’t find much. I guess it’s an architecture-level issue, then? I’ve switched to Chromium in an attempt to isolate the problem further but to no avail.

 


So, hey, a keep-alive. Between me not being able to keep my New Year’s Resolution (at least the part that tells me to write about my experiments here) and some other problems, personal and/or technical, I’ve been going through lately it feels refreshing to be actually writing in casual English instead of code.

See you around! ~Chad

  1. Though you may say that this needs further testing, I noticed that it works quite well in Linux but really annoyingly erratic in Windows. So yes, sentimental value and all it must be still fine. []
  2. The one with a Windows logo printed on it. []

There. Google has done it, again. That is, rolling out over-drastic changes to a product I’m already very familiar with to the point that it is alienating. And this time, they even changed the whole product, not bothering to stack the updates with an old brand.

In an effort to push Google+ to more of it’s users, Google has rendered Google Talk obsolete in favor of Hangouts—Google+’s video chat feature. I’m not allergic to change but it seems to me that Google has been pushing “social” to their users too much that they dropped the non-social use-cases.

My review for the Hangouts Android client says it all1:

I understand that Google made a lot of improvements for hangouts but they forgot their basics. From Talk to hangouts they removed status messages/indicators and the quick list of people who are “online”. Inconvenient as we use Talk in the workplace. Moreover, they now list your contacts based on your most recent conversations. Hardly useful if I just want to leave a message to someone not-very-recent. And that piles up fast since I use Talk a lot.

At work, when I switched my provided workstation from Windows to Linux, I had problems choosing a client for GTalk, since Google does not support a Linux Talk client. Sure the web interface that comes with GMail is OS-agnostic but it also comes with the inconvenience that my chats are all in a single tab and I can’t drag them around my workspaces. I would’ve tried Gwibber, the one that comes by default with Ubuntu, but I was too lazy to hurdle the set-up.

In the end, I settled for the official Google Talk Chrome plug-in. It did not fit all my needs but it does what the web client does without having to be in a single tab and I can lug it around my workspaces. In fact, if only I did not have to open Chrome just to sign-in/sign-out to it, it would’ve felt like a native desktop app.

That is, until Google decided to force feed Hangouts to everyone. I first encountered how Google does Hangouts via the Chrome plug-in. And I found it plagued with the same problems I later found in Android. Being that I cannot see a list of all my contacts via the plug-in, I now always have a GMail tab running at the far-left of my Google Chrome window for starting conversations2.

For a long time, I managed to resist Android’s nags of updating the Google Talk app in my phone to Hangouts. I even set the Talk app not to auto-update. And then came the most recent Android update from Sony. I should say that I’ve been loving Android updates ever since my XPeria Z first prompted me for one3. But this most recent update had the audacity to replace Talk with Hangouts. Argh. Google wins.

I’m now looking for another GTalk (as I will persistently call it) client in Android. I’ve tried Trillian, the first IM client I used, from way back when class presentations where planned via Yahoo Messenger conferences and not Facebook Groups. Unfortunately, Google won’t let me log-in with it; apparently the Trillian app does not hit the Google log-in servers directly but, rather, some “gateway” server which does the actual log-in for you. Google sees the IP of these gateway servers and labels them as suspicious sign-in attempts. Right now, still looking for another one.


The Google Score

Google, Google, I still think you are among those who tackle the most interesting problems and research topics but I think you’re pushing social too much into the face of your users.

Google products I still love:

  • Google Chrome
  • GMail
  • Android
  • Google Docs
  • Google Maps

Google products which I think has gone astray:

  • Google Analytics
  • Google Reader (or, rather, what they did with it; it was fine until they sent it the way of dinosaurs)
  • Google Talk Clients: Chrome and Android

And with the talks of Google’s famous 20% time perk dying going rounds, it really feels like Google has changed. A lot.

  1. Incidentally, the first app review I ever gave, out of pure ire. []
  2. Or hangouts, as Google would now want you to call them. []
  3. Unfortunately, I don’t think there’s much to update with my old Samsung Galaxy Y, so not much fanfare there. []

I was at AngelHack. Not as a competitor though; I was there for API support. Chikka (a.k.a., where I work right now) exposed its mobile charging API for the developers to use. The API somehow falls into my responsibilities so I was there.

But this post isn’t about the experience, nor about our API, nor about how we had to reconfigure our backend set-up just a few hours before the coders started hacking because of some “logistical oversights”. The point I want to make here is about something that is only remotely related to AngelHack. I want to point to this article from the thought-provoking Aeon Magazine which basically asks if we (developers) are worth the hype accorded to us. It makes for a nice post-AngelHack reading especially as AngelHack is basically the search for the possible next big thing for geeks.

As a footnote, I quote from this advice found from Learn Ruby the Hard Way1:

Programming as a profession is only moderately interesting. It can be a good job, but you could make about the same money and be happier running a fast food joint. You’re much better off using code as your secret weapon in another profession.

People who can code in the world of technology companies are a dime a dozen and get no respect. People who can code in biology, medicine, government, sociology, physics, history, and mathematics are respected and can do amazing things to advance those disciplines.

Really makes you think. Interesting food for thought.

  1. No. I am not actually learning Ruby. []