Dynamically loading modules in Python

Have you wanted to load modules, when you didn’t know their name?

I recently came across this problem while refactoring GoogleBot (my IRC bot). I wanted to place loadable modules in a directory, and have them all loaded, without having to know what they are named.

A good fellow called Issac at work helped me with this:

import imp
googlebot_mod_dir = os.path.abspath('modules')
sys.path.append(googlebot_mod_dir)
for module_file in os.listdir(googlebot_mod_dir):
module_name, ext = os.path.splitext(module_file)
if ext == '.py':
module_location = imp.find_module(module_name)
module = imp.load_module(module_name, *module_location)
globals()[module_name] = module

Works like a charm!

Clever advertising

I was standing around Ironforge tonight, about to log out of World of Warcraft, when gnomes started falling from the sky.

“Huh, that’s never happened before”, I thought.

As it happens, the gnomes were falling in a very particular pattern, and when they landed they died, leaving their bodies on the ground in pre-determined spots. Someone decided to do some very clever advertising, see the screenshot, below:

Wowmine advertising

I took several screenshots inside WoW, and then put them together quite easily using the free, open source program “Hugin“. It was pretty easy to use, I recommend it for photo joining (aka stitching).

Phil Gordon comes to Google!

Phil Gordon, WSOP final table, World Poker Tour champion and former host of Celebrity Poker Showdown came to Google on Friday.

We had our annual company picnic at the Shoreline Amphitheater, it was pretty decent. One of the highlights was a 150 person no-limit Texas hold ’em tournament. I don’t know how they pulled it off, but they managed get Phil Gordon to come and host it.

This was especially remarkable as Phil had been playing in the WSOP tournament until 4am the that morning. The winner of the tournament walked away with a chip set signed by Phil. It was certainly a great day.

Our new puppy

Ginger went to sleep about a week ago 🙁
Brandy and the kids were incredibly sad, so now we have a new puppy in our house!

You can read about her in Brandy’s blog, here.

Net Neutrality in the Senate

The Senate will be discussing it’s own version of the telecommunications reform act, which passed through the House last week.

The bill in the Senate differs slightly from the bill that passed the House. The Senate version currently has the FCC doing a study to determine if network operators are deliberately manipulating traffic crossing their networks.
This bill would leave it up to the FCC to enforce their own feeling on the matter, which would likely result in lengthy legal cases lasting years and costing the taxpayer even more money.

That aside, I’m not entirely sure I trust the FCC to be on the consumers side. Many times in the past they’ve shown they can be pretty unfair to organisations if they feel like it.
It may be interesting to note, that the FCC doesn’t have to provide much justification on its rulings when it decided for or against a case.

Scary.

The House is lost, on to the Senate

A little late, but an update as promised.

The COPE act did indeed pass the House, and the net-neutrality amendment was defeated.

The short of the story is, that several representatives argued that a neutral Internet would pass too much of the cost of building tomorrow’s networks to consumers.

Here’s what I would like to know:
The telecoms companies already charge their corporate customers  like Google and Yahoo, based how much bandwidth they use. If they use on average 50Mbit/s, they pay for that. If they use on average 25Mbit/sec, they pay less.

These costs were determined by the telecoms companies. They already know how much money they need from their customers in order to build these new faster networks.
Smart business owners understand that a part of their profits need to be applied to the upkeep of their establishment. Telecoms companies are no different!
AT&T alone expects to save $600m to $800m because of the merger with SBC late in 2005. Their first quarter profits were also up significantly compared to the first quarter of 2005. They also added over half a million new broadband customers, increasing profits there over 14%.

Since the start of 2003, AT&T’s stock price has remained at a fairly steady ~$26 average price. During this whole time AT&T has been constantly upgrading its infrastructure, so where has this sudden panic over upgrade costs come from?

Think of it this way: When you bought a top-of-the-line computer 5 years ago it could have cost you $2000. A top-of-the-line computer today would also cost you about the same. The same is true of most things, including the cost of building a faster network. The costs AT&T, Verizon and Comcast talk of are actually the cost they would have to invest in their networks anyway as part of their projected growth.
So you can see why a lot of this seems suspicious.

More this weekend…

Net Neutrality update

Tonight the House voted on the Communications Opportunity, Promotion and Enhancement (COPE) Act of 2006. This Act would create franchises of regions which were previously monopolised by cable companies. The bill would allow telecoms companies to much more easily sell cable programming to customers over their IP networks, for consumers to watch on their computers.

Attached to this was the amendment for network-neutrality that I spoke of in my last post. Unfortuantely while the Act was passed, but the network-neutrality amendment was defeated 269-152.

At the moment the next move is unclear to me, but I will find out in the morning and give an update. Please continue to get in touch with your representatives from http://savetheinternet.com/ , under the “Act Now” link.

Net Neutrality

This isn’t a widely read blog (yet), but I’m hoping that someone will read this and get the word out as best as they can.
In a few days, a bill will be presented in the House of Representatives, which could have quite serious and lasting repercussions on the Internet.
The bill has been lobbied by all of the major telecoms companies in the US as something they need to help them grow into the future.

In short, they are demanding to be allowed to charge companies like Google, Yahoo, YouTube, MySpace, and anyone else out there who offers multimedia content to their users to pay extra for it to keep the same quality of service that exists today. If a company such as YouTube does not cough up, people visiting their site will quickly find that they cannot access the videos easily at all. And even when they do, the content will take much much longer to download. If you think this isn’t bad, try watching some of these videos on dial-up for a week.
The argument put forward by the various companies, is that such multimedia content uses up a lot of bandwidth on their networks, networks which the telecoms companies built. They also argue that in order to continue growing, such that the Internet doesn’t become a large congested network, they need to invest in making faster networks and for that they feel the people using the networks most should pay up.

This seems reasonable until you realise there are a few problems with the stated arguments:

  1. Companies like Google, Yahoo, YouTube, etc already pay the telecoms providers a whole lot of money to be able to connect to the Internet. They may not pay AT&T, Verizon, et al directly, but they do pay their ISP, who in turn pays AT&T, Verizon, Comcast..
  2. The customers of said telecoms companies who have broadband also pay a monthly fee for their Internet access, and often pay quite a lot of money for this (my own bill is over $75 a month).
  3. The networks that were built by the telecoms companies, were heavily subsidised by US taxpayers. The companies now wany to take the subsidised networks and start making much more money from them, again at the cost of the taxpayer.

For many years now we’ve lived in a “neutral” internet – everyone had equal access and everyone was treated equally. Someone with a small DSL connection could host their own web server and it would be given equal preference compared to a large company.

This is not a partisan issue. Many members of Congress from both parties agree that network neutrality as it stands today is best of the Internet. It is how the internet has grown and how it should be allowed to grow, as a free-market. However, many are still undecided.
Please visit http://www.house.gov/ and contact your representative, and tell them that you disagree with the proposed bill.
You can also visit http://www.savetheinternet.com/ and express your voice there.
Eric Schmidt, CEO of Google, today put out an open letter to all users of Google on this matter. You can read it here: http://www.google.com/help/netneutrality.html

Shortening URLs in Mailman

Mailman is a great mailing list manager, and tonight I learnt how to implement my own functions.
Contrary to the FAQ, there is actually a really good way to extend the functionality of Mailman and it’s quite easy too.

Mailman uses python scripts called handlers, which are kept in mailman/Mailman/Handlers/. Each script contains a method (def, function, whatever you want to call it) called “process” which is automatically passed three arguments.

Basically in here you can do anything you would in a normal python script. The ‘msg’ variable that is actually an instance of a class which contains all of the message data (headers, body, and much other data). Once you’re done, put your new script in the above directory and add it to the GLOBAL_PIPELINE list in mailman/Mailman/mm_cfg.py (this file gets default from Defaults.py, copy anything you need from Defaults.py and put it in mm_cfg.py).
If you’re interested in doing any hacking of messages, look at Decorate.py in the above directory. I was able to make a very simple but effective URL shortener. Every message sent to my mailing lists now gets saved to a database and a new shorter URL is put after the original. If anyone would like to see the code, let me know 🙂

twisted.web and Method Not Allowed

Today I was developing a little application using Python’s Twisted framework. Everything was going as well as could be expected until I tried to POST some data to the application.

“Method Not Allowed
Your browser approached me (at /) with the method “POST”. I only allow the methods here.”

Googling for the answer, I found suggestions to add the following line to my application:
allowedMethods = (‘GET’, ‘POST’)
Unfortunately that doesn’t work by itself.

exarkun on irc.freenode.net/#twisted.web said my application was probably missing a render_POST method, which was correct. The problem was fixed by adding:
render_POST = render_GET
after my render_GET() method.

Posts navigation

1 2 3 4 5
Scroll to top