I did it again, I moved my blog. There are various reasons for doing so, but mainly to take load off of my private server.
So, my new blog can be found at http://techandsp.blogspot.com/.
I did it again, I moved my blog. There are various reasons for doing so, but mainly to take load off of my private server.
So, my new blog can be found at http://techandsp.blogspot.com/.
I know I wrote about Android already today, but there is another thing that concerns me right now. I am owner of an Android-based phone (an HTC Dream) and recently switched my mobile network provider. The problem is that my new provider is a virtual provider and as such there is no real network of that provider. Now Android has a feature to turn off broadband connections when in roaming mode, which itself is a great idea and can save you from paying quite a lot of money when the phone connects to 3G abroad, but this feature also turns off broadband connections when roaming locally. All this is being discussed in bug report #3499.
After noticing this problem I became curious on how Android detects that it is roaming and I found the GsmServiceStateTracker.isRoamingBetweenOperators method to be responsible for that magic, but soon noticed that the method is not only inefficient, but also doesn’t work as intended. This is hardly related to the bug mentioned above, but let’s have a look at the code in question:
/**
* Set roaming state when gsmRoaming is true and, if operator mcc is the
* same as sim mcc, ons is different from spn
* @param gsmRoaming TS 27.007 7.2 CREG registered roaming
* @param s ServiceState hold current ons
* @return true for roaming state set
*/
private
boolean isRoamingBetweenOperators(boolean gsmRoaming, ServiceState s) {
String spn = SystemProperties.get(PROPERTY_ICC_OPERATOR_ALPHA, "empty");
String onsl = s.getOperatorAlphaLong();
String onss = s.getOperatorAlphaShort();
boolean equalsOnsl = onsl != null && spn.equals(onsl);
boolean equalsOnss = onss != null && spn.equals(onss);
String simNumeric = SystemProperties.get(PROPERTY_ICC_OPERATOR_NUMERIC, "");
String operatorNumeric = s.getOperatorNumeric();
boolean equalsMcc = true;
try {
equalsMcc = simNumeric.substring(0, 3).
equals(operatorNumeric.substring(0, 3));
} catch (Exception e){
}
return gsmRoaming && !(equalsMcc && (equalsOnsl || equalsOnss));
}
Okay, let me summarize what this piece of code does wrong, at least from my understanding:
Now in my case my SIM card doesn’t seem to provide the phone with a alphanumeric identifier, so the first two comparisons always fail for obvious reasons and, looking at the inline-if in the last line of that method my phone will always indicate that I am in roaming mode, even when I am not.
The problem is not only the logic which seems to be wrong, but I rather see the inefficient comparisons used there to be a major problem in embedded systems like mobile phones. This is the first piece of Android code I have had a look at, but if all other code is as ugly and inefficient as these few lines Android really needs some major fixes. Related to this I have reported bug #4590 and forked the git repository in question over at github, to fix this method, should be a matter of 5 minutes.
I have been reading a great many posts about Android lately, some consisting of criticism, some of praise and some simply addressing issues in the Android “community”. Let’s have a look at those.
Matt Porter’s Android Mythbusters presentation and Harald Welte’s reaction
I haven’t seen the presentation live, but I had a look at the slides. Impressing work done by Matt putting all this information together. However, we all knew that Android only (ab-)uses Linux, without making use of the GNU userland for a long time, didn’t we?
In his presentation Matt has shown things such as Android’s udev “replacement” that uses hardcoded values for device node creation and (on his blog) Harald has then come up with a statement I have found to be very strong:
The presentation shows how Google has simply thrown 5-10 years of Linux userspace evolution into the trashcan and re-implemented it partially for no reason. Things like hard-coded device lists/permissions in object code rather than config files, the lack of support for hot-plugging devices (udev), the lack of kernel headers. A libc that throws away System V IPC that every unix/Linux software developer takes for granted. The lack of complete POSIX threads. I could continue this list, but hey, you should read those slides. now!
Now both of these statements target technical details, but the root of the problem seems to be elsewhere.
Where is my Android 2.0?
Okay, that heading might not be making any sense in the context of this post at a first glance, but let me elaborate on that. Google and the Open Handset Alliance refer to Android as being an “Open Source” operating system, but the project is different from “real” Free Software projects: development takes place in a closed group and the results are shared with the community later on, when they are deemed to be ready.
This means that innovation also takes place behind closed curtains and that the community is not involved in the actual development process at all. Lately we have seen the result of that, as Motorola is bragging about working close with Google on Android 2.0 (”Eclair”), but the AOSP source trees, open for everyone to have a look at, show no signs of version 2.0. In fact no changes that might even remotely suggest the release of a new major version have been made public in the past few weeks. So where is the openess there?
Actually, the Motorola Droid has already shipped with Eclair on 6th, but still, there is no indication that Eclair will be made available to the broader public.
In short Android seems to be developed behind closed curtains, with hardly (read no) community input whatsoever and is sometimes released as Free Software, not what I would describe as an open development process.
The Android Market problem
As we have seen in the past Google is enforcing their copyright on proprietary applications that ship with pretty much every Android device, such as the Android Market. This has become really clear when Steve Kondik received a cease and desist letter when packing the Google-proprietary applications into his ROMs. Okay, it’s Google’s right to enforce their copyright and there is nothing wrong with actually doing so, the thing I really have a problem with is something else: the Market is proprietary.
Now what this means should become rather clear. You can have an Android device without Google’s proprietary bits, but with default settings you just do not have any way of installing additional software. In my opinion the Market should be freed by Google themselves, or the community has to react and come up with a free replacement to overcome the vendor lock-in. Oh, you might know a replacement called SlideMe (or Mobentoo) already. Well, that bugger is proprietary too, so not a solution at all.
Nokia and Maemo to the rescue
In most discussions about the openness of Android someone throws in Nokia and Maemo, as a solution to the dilemma. Reading all those positive comments I simply had to give it a try, but all my hopes were destroyed within a few minutes.
Let’s start with the good news and let alone the reason why my hopes were destroyed for another minute or two. Maemo is based on Debian GNU/Linux and various Free Software components, such as GTK+, gstreamer, esd and friends. Most of the system is Free Software which is a good thing(tm) and reading all of this really got me into Maemo. Okay, some applications seem to be proprietary, but I am sure that could be fixed rather easily, so I could once for all use a truly open phone.
…and then came the SDK installer shell script:
#!/bin/sh
# Copyright (C) 2006-2009 Nokia Corporation
#
# This is proprietary software owned by Nokia Corporation.
#
# Contact: Maemo Integration <integration@maemo.org>
# Version: $Revision: 1110 $
Now there is one question you should ask yourself: Why would someone trying to promote his platform as being open make the *installer* script for its SDK proprietary? Come on, it’s an installer script, how much of your secret juice could be in there? What’s the problem with people modifying it and working on this installer script in an open development environment?
I had high hopes for Nokia actually doing a bit better than Google, but it seems they’ve failed to do so. It may be me overreacting, but a proprietary SDK installer shell script scares me enough not to install the SDK and have a look at it for now nor to think about buying a Maemo-based device in the near future. Please Nokia, either get the facts straight or provide us with a free SDK to your free & open platform.
So, in short, Google is bad at working with the community and creating a truly open development process, and Nokia simply fails in terms of not scaring off prospective developers for their open platform with the proprietary SDK installer. Do you have any solutions in terms of an open phone environment, apart from what OpenMoko has come up with?
I just installed google chrome on Debian. It’s really awesome that I can’t believe my eyes at first! I found that most sites I visited usually are well displayed in the layout. And the fonts in it are much better than in my iceweasel. From the screenshots down below, I can say that no one could tell I’m using Debian instead of Windows xp! Although the linux version of Chrome has been released for a while, I told myself that iceweasel is good enough with my collection of add-ons and Chrome is just another beautiful and fancy browser which doesn’t have much more practical value in everyday use. Now I found I’m WRONG! Except the "speed boom" on openning and loading web page, the fine layout and its simplicity is really fascinating! I really recommend it to others who still have doubt in Chrome. At least, it offers another good choice of browser to you. If you’re interested, you can download it here and installed it with "dpkg -i google-chrome-unstable_current_i386.deb". And voila, you can now enjoy it!
my chrome screenshot 1
my chrome screenshot 2
my firefox screenshot
I just installed Ubuntu Karmic Koala on my workstation and came across the problem of not being able to move/drag Gnome panels around in order to have the panels on my primary monitor.
On the Debian system that was powering the workstation before this was a non-issue as I could simply click, hold and drag both the upper and the lower panel, but this didn’t work.
So, after a few minutes of googling I came across an entry at answers.launchpad.net[0] and a blog post, but I cannot seem to remember the URL to that one. I can imagine that some of you might be having the exact same problem, so the solution is holding down the ALT, whilst dragging as usual.
[0] https://answers.launchpad.net/ubuntu/+source/gnome-panel/+question/264
I wrote a final report for this project before the end of GSoC. Now I’ve uploaded it to alioth. One can use the instructions in the last part of this report to re-build the system I had now. Also, I decide to upload the whole tar ball of this system to somewhere online. Since it is more than 600M, I’m not sure it’s ok or feasible to upload it to alioth too.
SO WHAT I HAD NOW?
I didn’t touch this project since the end of gsoc. The report I mentioned above list everything I’d done before. To give a whole picture here, what I’d done is:
*A mips3 clfs system working a Gdium and fuloong 2F.
*Some build-essential packages of Debian built and installed on such a system
SO WHAT TO DO NEXT?
What I need to do is :
1. Build all the build-essential packages of Debian and then using debootstrap(this may need further confirm from other experienced developers) to install it on fuloong or Gdium.
2. Build an apt repository to hold these .deb packages to enable the apt-get access from the debootstrap system.
3. Providing benchmark to measure performance of the old mips port of debian and the new system
SO WHAT IS YOUR PLAN?
I have a internship and some lab work now. Basically, I only have spare time at Saturday and Sunday. I think many developers of free/open source project have the same situation with me and they have done so many great jobs. However, since I had underestimated the difficulty of my project in the past, I decided to make a "loose" for now so that I could finish it little by little. So my plan is:
* December 31st, 2009 : before this time, I’ll finish the first target of todo list with quality.
* March 31st, 2010 : second target
* May 31st, 2010 : last
I agree with the point that free/open source projects have no end. So the process contains continuous improving. The plan is just a rough picture I can gave to others.
This is the first update since the final evaluation of my gsoc project this year. Unfortunately, I failed at last. I’ll keep my promise to finish this project. Plus, this project is not only mine; many people and communities hold high expectation from it and they really show their care by generously helping me with all kinds of support. So comparing to this, I really can’t stop working on this project simply becauseof my personal emotion stuff.
|
Sandyleo26’s, I’d like to add you to my professional network on LinkedIn. - Sha Every day, millions of professionals like Sha Liu use LinkedIn to connect with colleagues, find experts, and explore opportunities. © 2009, LinkedIn Corporation |
Debconf9 is over for more than one week but th
I’m very lucky to get full sponsorship from Debian and Google for Debconf9, which was held from July 23rd to July 30. Although my preparation for this task was full of frustration and time&cost-consuming events with local Consulate, I still made it in the end. I want to thank many people for supporting me all the time, including but not limited to Arthur, Steve, Anto and of course my Mom.
The Debconf was great. There were many lectures and BOFs everyday. Actually, you couldn’t listen to them all because many of them were started simultaneously at different room. But thanks to the video team, I can watch the stream of a event while stayed at different room listening to another lecture etc.Of course, my favorite were those lectures about emdebian stuff because that’s closely related to my project. I benefited a lot from attending these BOFs or lectures which gave me much more knowledge than I can digest. Also, I found my poor English listening improved. 
I gave a 3min talk on my project, too. It was very short but still attract several peoples attention to try to help me. 
Besides these talks, we had day trips, parties and talks over meals which were all for knowing each other and having fun at making friends. What’s more, I made many friends by playing table-tenis with them. One of them is Francisco. I spent my last two days in Madrid, living at his apartment. He showed me around at Madrid and we got home the day before I left so late that I overslept the next day and missed the flight. Luckily I had him helping me ask around in the airport and finally got a ticket home.We both agreed that never sleep late if there’s flight tomorrow. 
Debconf is over more than a week now and I began to miss the life there:-)
by slackydeb (noreply@blogger.com) at August 06, 2009 09:43 PM
by slackydeb (noreply@blogger.com) at August 06, 2009 05:06 PM
by slackydeb (noreply@blogger.com) at August 03, 2009 10:25 PM
by slackydeb (noreply@blogger.com) at August 03, 2009 01:24 PM
by David Wendt (noreply@blogger.com) at July 30, 2009 03:52 AM
by David Wendt (noreply@blogger.com) at July 27, 2009 04:34 PM
If you are in Cáceres for Debconf and happen to fix a bug (any) in Debian, look around for me and I’ll give you a quite incredibly most awesome too damn good peruvian chocolate. It’s amazing, you have no excuse to not fix some extra bugs.
Extra surprise even more omg awesomeness bonus for the most evil bugs and release-critical ones.
by slackydeb (noreply@blogger.com) at July 22, 2009 03:28 PM
by slackydeb (noreply@blogger.com) at July 20, 2009 01:35 PM
by slackydeb (noreply@blogger.com) at July 20, 2009 01:04 PM
by David Wendt (noreply@blogger.com) at July 18, 2009 12:48 AM
by slackydeb (noreply@blogger.com) at July 14, 2009 08:47 AM
It’s been some time since I re-installed Debian over my Kubuntu install, so I thought I’d discuss some reasons why I changed back to Debian, what my experience was like, and some learning opportunities.
One reason I made the switch was because there was a utility newly packaged for Debian, Frama-C, which was not available in Kubuntu at the time. It also frustrated me that I was having various frustrations with the installation, not the least of which was an unreliable and quite crashy KDE Plasma.
When I reinstalled this time, I picked the normal install but told it to install a graphical environment, which gave me a GNOME desktop environment. I actually rather like it, the setup didn’t ask too many questions and everything was set up perfectly. There was some minor tweaking, but it was all done by the easily accessible System menu and all the applets therein.
Now, I wanted to be able to use the server both as a virtual machine and as a physical dual-boot. This wasn’t working properly with GRUB-2, so I had to stay with version 1.96, which works rather well. I even spent some time making a pretty splashimage for it, which looks rather nice, even if I don’t see it all that often.
If I boot into the Virtual Machine, all the hardware is detected properly, and there aren’t even complaints about the fact that a bunch of hardware disappeared — certainly very good news if you decide to do something like move your hard drive to a different machine. Likewise, if I boot into the desktop, everything works well there too.
One issue I came across during the installation was having to teach Network-Manager how to configure my network interfaces. In my VMware NAT setup, there is no DHCP server, so the IP address, subnet and gateway information needs to be statically defined. Luckily, Network-Manager was able to do this based on the MAC address of the adapter — inside my virtual machine, it had a VMware-assigned static one. Through this, Network-Manager had an easy way to determine how to configure my network, and it works beautifully for Ethernet and Wireless (when Debian is running as the main operating system) and also for VMware NAT (when inside the virtual machine container).
Anyway, I have now been developing quite happily inside a Debian + GNOME desktop environment. The system runs fine even within a constrained environment, though I miss KDE’s setup with sudo; with GNOME, the option seems to be to have the root password entered every time privilege escalation is necessary. I don’t like using a root password — on my server system I don’t use the root password at all, and do everything I need to do via sudo. It’s okay for me because I log into the server with a private key and have disabled SSH password authentication for security reasons.
One thing that is still weird for me is that my system currently shows a time of 01:53 even though it is 23:57 in my timezone. Presumably the few minutes of difference is because the virtual machine clock and my system hardware clock aren’t synchronized perfectly, but more than that, I think it’s an issue with the Date applet somehow. I haven’t looked into this because the thing is running inside a virtual machine, so it doesn’t bother me much.
I have looked high and low to see where to change the time zone, and to my knowledge the system knows that it’s in the America/Toronto time zone. The house picture next to Timmins (the city I am in right now, though it doesn’t matter since the timezone is the same) seems to indicate to me that it’s set to the appropriate time zone.
I think it’s due to VMware synchronizing the virtual machine clock with my host machine clock. Windows (my host operating system) stores the time in the local format, which I believe Linux thinks is UTC. Still, it doesn’t explain the weird display it’s got going.
Someone noted last time that I didn’t make direct mention of which programs are only offered on Windows and not on Linux/etc, and that do not have reasonable replacement on these systems. Kapil Hari Paranjape noted that I was sounding somewhat like a troll by simply saying that I don’t think Linux is yet ready to replace my environment. Here was my reply:
Far from a troll, I’d really like Debian and Ubuntu, but moreso Linux in general, to improve at the pace it has been doing so. It’s made great progress since the last time I tried it out on my desktop, but I have to acknowledge that there are lots of rough edges right now that should be worked out.
One of the advantages of huge proprietary development organizations like Microsoft is that they have tons of developers and can implement new features at a relatively quick pace, even if they’re half-assed. Developers’ pride in the FOSS community prevents this overly quick pace of development in favour of more secure, more stable platforms. Which is a good thing, I think. But nonetheless it results in a “slower” development pace.
The applications I’m complaining about are things like:
- SolidWorks (a CAD tool for designing parts and assemblies, used in manufacturing and mechanical engineering)
- SpectrumSoft Micro-Cap (a version of software similar to PSpice used by my school)
- AutoCAD (another CAD tool)Luckily this is changing, but only for the large & most popular distributions:
- MathWorks MATLAB (runs on Linux and Solaris, etc.)
- Wolfram Mathematica (which has versions for Linux and MacOS X)
- FEKO (runs on Linux and Solaris among others)Anyway, I still consider SolidWorks to be a rather big program not supported on Linux, which is a big issue for those working on Civil Engineering programs. There are most probably others which are very domain-specific that I don’t even know about.
There is a nice matrix comparing cross-platform capabilities of CAD software: http://en.wikipedia.org/wiki/Comparison_of_CAD_software
Oh, one final thought: perhaps that KDE Recommends: should be moved to a Suggests: instead, on account of its heavy dependencies, requiring mysql-server installed on desktop machines.. WTF!
Oh, and on another note, I re-installed Debian using the non-expert Auto Install and it installed Gnome rather flawlessly, much like installing Ubuntu, which was pretty nice. So kudos to those who have been working on the main installer; it seems as though the advanced ones really give you some rope to hang yourself with, though :-)
Oh, and k3ninho told me that there is an initiative from the Ubuntu community called “100 Paper Cuts” to help fix small bugs like those I was complaining about. I hope this leads to an improved user experience, and I’d really like to see some of those changes propagated both upstream to Debian and upstream to the KDE folk.
During my install of Kubuntu + KDE, I felt that plasma was crashing more than Windows Explorer — it felt like the days when I was running Windows ME, and the shell would crash, and the system would restart it. Repeatedly. It’s exactly what seemed to happen with plasma. I’m not sure if it was something I screwed up during configuration (presumably so), but KDE was far too complicated for me to try and debug. It might also have been a result of me running my system within a fairly constrained virtual machine environment – the system only gets 768MB of RAM and no access to the actual graphics processor unit (since it’s virtualized).
Posted in Computer Science, Software Engineering, User Friendliness Tagged: Debian, GNOME, Google Summer of Code, KDE, System Administration, User Experience
So finally I have the time to provide you with a weekly update, instead of my usual bi-weekly ones.
Unfortunately I did not work on anything on last week’s TODO list, but found other issues I worked on and corrected. So let’s have a look at what I’ve done.
Debian packaging update
I have done some work on the Debian packaging, which allows update-manager to be built using dpkg-buildpackage now. The way packages are splitted is not finalized yet and not up-to-date with my (and my mentor’s) idea of how we should do that. You can expect an update to that soonish.
Automatically invoking package list reloading / update check
There is a command line switch (namely -c, or –check) now, that automatically performs an update check on startup. This gives other programs, like software-properties, a way of forcing a check when, for example, the package list sources have changed.
Checking/unchecking all updates in Gtk frontend
Finally the small feature of selecting or deselecting all updates works in the Gtk frontend. Special cases like “all updates already checked” or “no updates checked” yet are handled too, meaning that you can only use one of these methods if it actually makes sense.
Package dependencies in python-apt backend and Gtk frontend
Both the python-apt backend and the Gtk frontend are now aware of package dependencies. This means that when you select an upgrade that depends on another one that other update is selected too. The same works vice-versa too. Additionally the UI now lists all dependencies and dependencies on packages that are not installed yet and automatically deselects all updates that would requires new packages to be installed.
Displaying of overall download size in Gtk frontend
There has been a missing feature (ok, maybe a bug) so that the displayed download size would not be updated in the Gtk frontend. This has been fixed.
Install button being set sensitive correctly in Gtk frontend
In the past the install button would be set to either sensitive or insensitive at startup and not updated afterwards. That means if there were no packages to update when starting update-manager, then checking for updates where new updates are found, the install button would not be set sensitive again. I fixed that too.
Sorting of packages in Gtk frontend
In the Gtk frontend packages were not sorted at all, which meant that finding a specific package was rather hard. I added code that sorts the update list by package name now, which solves this issue.
Bugfixing humanize_size
The humanize_size method, which is responsible for human-readable size displaying in the Gtk frontend contained a major bug so that sizes were rounded. Again, I was able to solve this.
Next week’s TODO list
As I didn’t find time to work on last week’s TODO list my new TODO list is in fact my old one, with additional “Bugfixing” and “Debian packaging” tasks:
The next thing you can expect me to update is the Debian packaging and the documentation, which are my highest priority tasks for now, followed by support for downloading and installing updates.
Happy hacking!
In these two weeks, I focus on two points in the timeline:
a) Write code to add the history to a sqlite database, and code to retrieve the history from the database.
b) Show the history of a particular package in the GUI and terminal.
I had thought about doing this after the Mid-term evaluation, but Daniel suggested me to first work on this before implement the new undo and the redo.
For a) I redefined the database draft that had been around for some time to have a database schema like this:
History Entries
/ \
Dpkg Changes Aptitude State
In which History Entries is used to store basic information about a change (package name, date, type of change, etc.) and the rest of the details can be subdivided into tables, and depending on the type of change, use some or other tables. Similarly, the retrieve of the history from the database depends on the type of change.
In b) I added command line support, thus having the ability to view history of a package as $ aptitude history [PACKAGE ...].
For the GUI, I have based on the code made to view the files and folders (filesview), and added a subtab in the packet tab.
Following the Filesview scheme, I created the HistoryColumns class to manage columns containing historical information, the HistoryTreeView and HistoryView classes with which were implement to handle the ListStore to view the data.
For now the details shown are few, but I hope to get the proper view of the related changes (deps, suggested, etc). Besides working with the cwidget/ncurses interface in future.
Gtk Viewer
Apart from that, I was in Bogotá a few days doing the paperwork for the visa, I get the answer the next Tuesday .
The outline for the DebConf could be something like this:
Introduction
Deliverable 1: History Explorer/Viewer (What has been done so far and what is missing)
Deliverable 2: Undo – Redo (What has been done so far and what is missing)
I hope we meet in Caceres.
Firstly I have to apologize again for not providing you with weekly update #4, but again I didn’t have the time to write one, so this post is going to sum up everything that happened since my last update.
Let’s have a look at my previous TODO list:
Documentation
Even though my TODO list entry contained a more detailed entry I have updated the UpdateManager documentation as a whole, leaving only a few blank spots right now.
Ubuntu distribution specific code
I implemented changelog fetching for Ubuntu, which works just as fine as its Debian counterpart now.
More unit tests
There are plenty of unit tests now, but not everything is being tested yet. I am especially proud of my Python interface validation code, that is being used in unit tests to check if handlers implement an interface correctly.
Update list downloading
Checking for updates is what caused me major trouble in the past few days. Basically I had all the code ready, but for some reason the UI froze, with no apparent reason.
However, today I was able to finally identify and fix the problem. As I expected my code was just fine, but python-apt was messing up. I am going to discuss the exact problem and its solution later on, but first: a screenshot. 
Note: As you probably noticed I replaced the default progressbar with a pulsating one, because we cannot get exact information on how many items/bytes to fetch and would likely get a progress bar moving backwards, which isn’t beautiful.
Further changes
The TODO list was rather short and I did a lot of other work, which I want to elaborate on.
Dynamic selection of frontend, backend and distribution specific modules
Even though this is probably not of any interest to John Doe, it helps a great deal when debugging code as all three components can be selected via separate command line switches now.
Additionally some magic has been put in place that automatically detects the system’s distribution and loads the corresponding distribution specific module. This is done via lsb_release and the newly introduced code in UpdateManager.Util.lsb.
Pylint cleanup
Just out of curiosity I decided to start a pylint run on the codebase and quite a few problems were detected, which I then fixed. To be honest though I added quite some code afterwards that probably needs pylint checking and fixes again.
update-manager IPC
My original plan and IPC design involved using callback functions and passing them between the different modules. Even though this worked out fine I had the feeling this wasn’t clean enough and decided to ditch this approach and replace it with handler classes.
The handler base classes now provide an interface of methods that are called on certain events and their implementations act accordingly. The main benefit was that I could easily drop a lot of enums and rather have different methods handling different events.
Gtk, threads and python-apt
With the new IPC approach it became easier to use threads that do the actual work in the background, which I had implemented in next to no time, but a few problems showed up.
Whilst cache reloading from within a thread worked just fine checking for updates did not, and until today I didn’t know why. I spent a good amount of time debugging this issue, even using python profiling, but nothing obvious showed up. The background process was running, whilst the UI froze.
Today I finally found the root of the problem: python-apt. Even though I assumed that the python-apt worker threads must be stealing CPU time from the thread running gtk.main I wasn’t sure how this could be happening, having two completely independent threads.
Now, the cause of all this mess was that Python has a global threading lock and it seems as if this one is *LOCKED* when running C-code, such as the one python-apt comes with. The solution lies in calling Py_BEGIN_THREADS_ALLOW and Py_END_THREADS_ALLOW from within the C code, to release the global lock and let the Python interpreter do some work every now and then.
As with the python-apt acquire code I was able to allow other threads to work as soon as the fetching code starts working and only disallow threads when actually modifying Python objects or calling methods and/or functions. Surprisingly python-apt already made use of this in its cache loading code, but not the fetch progress code.
Fixing this problem took me less than half an hour and you probably can’t believe how glad I was to finally get things working again.
UI updates & other changes
Some details in the UI were anything but optimal, like horizontal scrollbars in a few places, which I removed. Additionally I saw the need to move some code out of the Gtk frontend’s __init__.py file and to a separate ui.py file.
A full list of all changes I made is available from the bzr changelog at bzr.debian.org.
A few more screenshots
Finally, I would like to provide you with two more screenshots (don’t worry about my system being insecure because of not applied updates – this is a testing machine that is not up-to-date on purpose):
TODO list
My TODO list for next week:
Recently there was a thread on the Google Summer of Code students’ list discussing gender dynamics in open source, but more broadly, interactions between those of different genders (mainly the discussion was simplified to be a discussion of sexes, which I think demonstrates the lack of understanding of the difference between gender and sex. But I suppose that’s a blog post for another day).
It was noted that many of the women on the list have blog addresses and other details that quickly self-identify the authors as female. There was discussion about whether this is a good thing or not, and the possible reasons behind it.
Here is what I wrote:
I think what you mention about yourself shows the world what you think about yourself, and what you consider yourself.
If first and foremost you associate your identity with being female (or male) or straight (or not)… then I guess that’s your prerogative.
But I, for one, am not /just/ an Asian male. I’m not just a Computer Science student. I’m not just a coder. I’m not just an Engineering student. I’m not just 20-years old. I’m not just a blogger. I’m not just an Open Source contributor. I’m not just an advocate of strange and often unpopular ideas.
I am a human being, with many dimensions. And I don’t try to simplify it by putting myself in a box and categorizing myself as anything.
I think that the key is just to understand everyone for who they are, and part of that is being somewhat ambiguous. As Leslie [Hawthorne] somewhat alluded to, it’s about managing people’s preconceptions about you.
I do not actively try to hide that I am male, or that I am Asian (you might guess that from my last name). There are all sorts of preconceptions people might have about things, and there are lots of -isms we should seek to avoid. (I’m Asian – maybe that means I’m a bad driver, and that I can’t pronounce Rs. I’m male – maybe I’m violent. I’m in Computer Science, presumably that means I play Dungeons & Dragons with my classmates on the weekends. I’m in Engineering, maybe that means I’m sexist.)
The reality is: none of these things should matter, nor should they define you.
Just be yourself. You show to the world what you consider relevant about yourself.
And for what it’s worth, I found out the other day that someone I respect and admire in the open source community is a teenager. Somewhere around 15 years old. It’s impressive, really. I look up to him, because he’s a really smart guy. But that wasn’t something he brought up right away; his nickname wasn’t “smartdude15″ or anything
like that. That’s the magic of open source, and the Internet — I judged him purely on his knowledge. And once I did find out, I thought to myself… Wow, would I have thought the same thing of him if I knew his age right away? Would I have even given him a chance, or would I just dismiss everything he said as something an immature teenager might say?
I think along with sexism there are tons of other issues to worry about, like racism (consider how difficult it is in some cultures, and even in Western culture, to be really accepted if you are gay, lesbian, transgender, bisexual, two-spirited, asexual, intersex…) In fact, being gay was considered a disease until relatively recently.
I’m glad for all the progress women have made in the past several decades. Not everyone has reached a point where they are accepted in mainstream society, and not everyone feels comfortable announcing certain details about themselves.
If *all* you are is a woman in a male-dominated world, then I feel sorry for you. I truly, truly do. Because none of the women I respect and admire are that. They are, first, talented Engineers, Scientists and Programmers, who are only incidentally female. Being female isn’t something that really identifies them any more than the colour of their skin, hair or eyes. No, no, they are talented, and that is, in the end, all I care about, and that is one reason I am grateful for Open Source — because you oftentimes don’t meet the people you are working with all the time in real life, so you cannot judge them on anything other than their ability.
Posted in Computer Science, Engineering, Peer Relationships, Relationships Tagged: Community Dynamics, Equity, Gender Discrimination, Google Summer of Code
by David Wendt (noreply@blogger.com) at June 24, 2009 09:13 PM
When I started working on update-manager I thought using zope.interface for my interfaces was a good idea, but soon realized that it lacked a way of actually validating a given interface against an implementation. The only thing it did was checking whether the implementation defined that it implements the interface.
Now, whilst writing some unit tests for update-manager I came up with a simple way of doing “real” validation, and I would like to share that Python code with you.
Firstly, I’d like to give you an overview of which checks my code carries out:
I consider at least the first and last check viable for validation of an interface against its implementation. The second check I listed is not that useful, and may produce false positives when someone uses certain decorators, I did not carry out any tests on that myself though.
The code can be found in update-manager’s repository (link) and (for now) is licensed under the GPLv2 or later. I am willing to distribute this code as a separate Python module (maybe under a more permissive license like the LGPL) if enough (let’s say at least two) people are interested in it, so please let me know if you like it.
Apart from the code itself the unit tests in the file linked above should explain how this beast exactly works.
Happy hacking!
by slackydeb (noreply@blogger.com) at June 22, 2009 07:36 AM
by David Wendt (noreply@blogger.com) at June 22, 2009 05:00 AM
Hi everyone,
Sorry for not showing up at weekend meeting. I hope you all enjoyed that:-). First let me summerize what I’ve been busy with in the last two weeks and the status of the project.
Abstract:
1. Make a self-build minimal linux system running on fuloong2f box.
2. Building binutils, gmp, mpfr under mipsel on fuloong2f. However, it turns out I use some wrong flags and option when building them
3. Learn zhangle’s(r0bertz) loongson overlay; Learn bash and vim. Learn stuff about automake, autoconf and libtool.
4. Play a little with Gdium Liberty 1000.
5. Applying visa: Make appointment with Spain Consulate on next Tuesday and asking Anto for invitation letter(still not received).
Details:
1. Last night is the most exiting time since I’ve been accepted in this year’s GSoC. I’ve finally built a N32 mipsel system and run it on my fuloong2f box! Last week, I’ve talked with zhangle(r0bertz) on how to create such a system. I could either directly build packages on debian-mips on fuloong(because I’ve installed multilib tool-chain on it) or build packages on a x86 maching using the clfs method. Later, I found this[0] which use the second method. This’s a very detailed guide and due to the fact that building packages on fuloong2f is relatively slow compared with my laptop, I choose the second method. The only drawback is this n32 system is not Debian and thus has no Debian specific features!
2. When I tried to build binutils on fuloong, an error "invalid asm constraint" happened. Fai point out several links to this problem and r0bertz gives me the patch in his loongson overlay. This overlay is, from my point of view, a collection of patches and scripts r0bertz has created when building gentoo on loongson. But I have little knowledge with gentoo and the patches in it have few instruction on when or why to use them, so I guess I’ll bother him a lot in future.
3.I’ve received a Gdium Liberty 1000 netbook[1] a week ago. It has a system based on mandriva running in it. I play aroud it a little until I failed to install the pppoe which I use to connect to my ISP. I plan to install debian on it when I have more time to kill. If you guys are interest in getting a developer’s version of this netbook you can visit[2].
4.The visa stuff is really A BIG HEADACHE for me. The Consulate finally agree to give me a interview on next Tuesday but Anto still haven’t emailed me the invitation letter. If I still haven’t received the letter by tommorrow, I’ll definetly impossible go to debconf9.
Plans next few weeks:
1. Make the N32 system "more debian".
2. Upload it to ??
3. Building more packages like X
4. Start writing documentation and prepare for mid-term evaluation
[0]http://zdbr.net.cn/download/Loongson64-2.0.htm
[1]http://olph.gdium.com/wiki/doku.php/hardware:start
[2]http://olph.gdium.com/
[tag GSoC,loongson,mips,n32]
Given this, the students have expressed in various ways against his re-election. Protests which the only way out that saw the direction of the university was the cancellation of the semester, extent to which students lost all the money and time invested. With the cancellation of the semester, 19 students took the main cathedral of the city and began a hunger strike and a large part establishes safeguards against the university for denying the right to education.
Yesterday the Board of Governors of the university decided to reverse the agreement that canceled the semester, but still maintains the Principal in his post, which is no solution to the crisis at the university.
I think I deviate slightly from the topic, I just wanted you to know a little the problems of the university, which is a reflection in small scale of the country, where the dictator of the moment simply to disqualify the opposition branded they as terrorist and sent to silence by force and not arguing as should be.
Since the previous report I had a more fluid communication with Daniel, in large part by the access to the repository. Most of these two weeks I have spent the time polishing the code that I made the first two weeks. Now I get the most information from the changes in dpkg and aptitude states, but recently I have been a problem presented in the order in which these data should be collected when I remove a global variable that was clearly a sign of bad programming, I am focused on circumventing this problem before proceeding. Another important change was the creation of a class to manage the database and the text logs.
For the next weeks I hope to start working on the undo and redo systems.
First of all: yes, I skipped update #1. I was rather busy with some assignments and exams at university and didn’t work that much on update-manager the past two weeks.
Anyways, this update contains everything that has happened since update #0.
Changelog fetching
The changelog fetching code has been added to update-manager. This means that the changelog will be shown in the details section now and should look the same it looked before. However, I have only written that code for Debian so far, but the Ubuntu part is on my TODO list.
Documentation
The documentation has been updated and uploaded to alioth and can be viewed here. I have set up a python environment on alioth which allows building the documentation directly, rather than building it locally and uploading it then. Basically this works by having a separate python packages directory, containing some mock modules that are needed (think gtk and friends here), allowing us to build the docs without having to install all dependencies.
I am planning on elaborating on this method and how to create such an environment in one of my upcoming posts, so stay tuned if you could use something like this too.
Additionally to this environment the documentation has been updated a great deal, including more modules and containing documentation for previously undocumented methods and classes.
Application module
I have reworked some aspects of the UpdateManager.Application module, allowing me to do unit testing on pretty much every aspect of the class. The problem I fixed here is that Application directly called sys.exit when something went wrong and now raises exceptions, which contain the status code and are handled in the respective scripts (ie. “update-manager”).
Gtk Frontend and updates from another thread
One thing I fixed was the problem caused by the changelog fetching code running in a separate thread and invoking a callback function that updates the UI. It seems as Gtk isn’t that happy when you do this and the UI wouldn’t be updated immediatly (it seemed that this only happened after some events, like scrolling the update list). This has been reworked and the callback function now checks if it was called from the main thread or not and calls gtk.gdk.threads_enter/_leave accordingly.
Changelog Viewer
After finishing the changelog fetching code I added the ChangelogViewer widget from previous update-manager versions again, supporting creation of links to launchpad and debian bugs (ie. LP:NNNNNN and Closes: #NNNNNN are now links) and displaying the version number in bold, among other things.
Weeding out UpdateManager.Frontend.Gtk.utils
Initially I just copied over the utils module from old update-manager to the new implementation, leaving every single function in there, but now I decided to weed out the module. The result is that only the functions actually used by this implementation remained in there. Related to this documentation of that module is pending and on my TODO list.
Version number
After a chat with my mentor we decided to bump update-manager’s version to 0.200-pre. This should make it easier to distinguish from the old version and indicates that a lot has changed. The first release following the -pre series will be 0.200.0, which should then include all functionality old update-manager included.
My TODO list for next week
Ordered by priority
by David Wendt (noreply@blogger.com) at June 18, 2009 10:18 PM
by slackydeb (noreply@blogger.com) at June 17, 2009 05:16 PM
For my Google Summer of Code project, I have been working with PerlQt4 bindings, which requires that I have Qt4 installed. While this is technically possible under a Win32 environment. Lots of people in the free software community vehemently oppose Windows, but while it has its flaws, I think overall the hardware support is still much better than Linux. True, this is because of Microsoft’s shady business practices, and because many companies keep their driver source code closed. I’m still using Windows XP Professional and quite happy with it, stability-wise and feature-wise.
As an Engineer, many applications we use on a regular basis are simply not available on Linux. They’re simply not replaceable with the current state of open source software, though there is some great stuff out there. Nonetheless, we’re still far from a point where engineers in general can switch to Linux — the application support is as important to an operating system as the kernel. Linux would be nothing without GNU’s binutils, for example.
I tried to install Debian first, as this is an environment I’m very familiar with. I use Debian on my development server, and it has worked wonders there. But everything I do on that server is command-line stuff. When trying to install a desktop environment, I followed the KDE Configuration Wizard, which isn’t too bad, but it expects an Internet connection throughout the process. The problem was that I didn’t have enough Ethernet cables to have both the desktop computer and my laptop plugged in at the same time, even though I had a wireless router set up, which meant I had to unplug the computer while updating packages, etc. Some of the updates took quite a bit of time, which was inconvenient for everyone else.
I eventually got the system to install, and told tasksel to set up a desktop environment. It was installing stuff, I typed ‘apt-get install kde’ and assumed everything would Just Work. After installing a whole bunch of stuff (which included a local install of mysqld, on a desktop machine?! — turns out it was due to one of KDE’s recommended packages, it starts with an A, I forget which). Anyway, then the environment didn’t “just work” as I had expected. Upon booting up my system, it just dropped me to a command line prompt. Fine, I thought, I’ll just use startx. But that was broken too. So after another few hours of fiddling I just gave up altogether.
While trying Ubuntu (the last time I had done so was probably in version 7 or so), I downloaded a recent image of Kubuntu 9.04, the Ubuntu flavour using KDE as a default desktop environment. It’s surprising that there has been lots of progress in Ubuntu and Linux in general. I have found that driver support is much better than it used to be, as it now detects my network card – a Broadcom 43xx chip – and does everything it needs to do. For the most part, my operating system “Just Works.” Great. This looks like something I might be able to slowly transition toward, completely replacing Windows except inside WINE or a Virtual Machine container.
Has Debian and Ubuntu made lots of progress? Sure. I can definitely see that Ubuntu is geared a lot more to the average user, while Debian provides bleeding-edge features to the power user. Unfortunately, despite being involved in packaging Perl modules for Debian, I fall into the former category. I’d really just like my desktop system to just work. Oh, and dual monitor support out-of-the-box would be nice too — I hear the new KDE and Gnome support this.
One thing Windows handles rather well is changing hardware profiles – when my computer is connected to its docking station, a ton of peripherals are attached. When I undock, they’re gone. Windows handles this rather gracefully. In Kubuntu, I got lots of notification boxes repeatedly telling me that eth2 was disconnected, etc. This sort of thing is undecipherable for the average user, so I’d really just like for these operating systems to be more human-friendly before they are ready for prime time on the desktop.
Posted in Computer Science, Customer Relationships, User Friendliness Tagged: Best Practices, Debian, Engineering, Google Summer of Code, KDE, Support, User Experience
by David Wendt (noreply@blogger.com) at June 17, 2009 12:55 AM
by David Wendt (noreply@blogger.com) at June 16, 2009 06:25 PM
by David Wendt (noreply@blogger.com) at June 12, 2009 02:45 AM
I can’t help myself learning Vim today. I found much fun and more magic hidden in it!
1. Enable mouse
:set mouse=a
2. tab pages
Yeah, vim have tab navigation! You can open a tab by :tabedit to open a empty file in a new tab or :tab split to open the current page in a new tab. If you have mouse enabled(see above), you can navigate among tabs with mouse just like under windows. double-click on the empty space will open a new tab and clicking the "X" on the right-most corner will close the current tab. With keyboard, :gt and :gT will go tab previous and next. Use :tabpage for more
3. Completion
In command-line, use the <Tab> key and <Ctrl-D> to complete commands. In insert mode, you can use <Ctrl-P> and <Ctrl-N> to complete your input. More in usr_24.txt. You can type :help and search the text "usr_24.txt" to get in.(But is there better way to go there in command-line?).
4. manage your window
:split and :vsplit will split your current window in two horizonally and vertically. You can move your cursor among these windows with <Ctrl-W> hjkl to move left, down, up and right.
And if you feel boring with so many windows, use <Ctrl-W><Ctrl-W> to move your cursor in the window you want to keep and press <Ctrl-W> O to close all other windows!
5. fold
I’m not quite familiar with this magic but it’s kind of very useful. In visual mode, selecting several lines and type :zf to fold these lines and :zd to delete the fold(Note: not the text). You can also just :zo to open it and after viewing :zc to close it.
To be continued…
–
Best,
Sha Liu
by David Wendt (noreply@blogger.com) at June 09, 2009 08:36 PM
Due to the inconvenience I came across when I play around with config files, patch files, scripts etc, I decide to manage at least one main-stream editor proficiently. And I choose Vim because to the learning process with vimtutor is really a fun to me!
Some How-tos I found useful:
1. How to display with/without color?
Type ":syntax on" or ":syntax off". To change the color theme, type ":colorscheme" and <TAB> to complete your choice
2. How to copy/paste from vim to other application?
There are 2 registers in vim to hold the selection "* and "+. The first is for current selection(which e.g. you select in visual mode) and the second is for real clipboard. Both can be used to exchange info. For the current selection, e.g,
"*yy to yank a line and then "*p to paste it in vim or press the middle button of your mouse in other application. Of course you can just select in visual mode and vice versa
For the real clipboard, e.g,
*yy to yank a line and then "*p to paste it in vim or CTRL-v in other application. It’s just the normal clipboard like in windows.
3.How to move quickly to a word in one line?
Use the "f/F" and "t/T" and repeat it with ";" and ",". This is much faster than pressing "l" and "h".
4.How to get help?
Type":help [keyword]". You can use "CTRL-O" and "CTRL-]" to jump back and forth with hyperlinks or any word. Vim’s help system is very smart and friendly if you can spend a little time learning it. If you have questions, always try to figure it out by turning to help. You’ll find eventually you can solve all the problems on your own
To be continued…
I spent my whole day building the binutils, luckily for me I finally succeeded in the evening. I think I’d better write it down for remembering.
Purpose : Since I need to create a N32 port for Debian, all that I need to do is to building every package(hopefully) in Debian repo from source, with the specific CFLAGS="-march=mips3(or loongson2f) -mabi=n32". And before that, I should have this set of cross-compile tool-chain. And Binutils is the very first package to be built.
Goal : build Binutils-2.19 using the multilib version of gcc-4.4 with parameter "-march=mips3 -mabi=n32".
What I have done:
1. Download Binutils-2.19.51 from cvs with this patch since it’ll later be used with building the kernel with the option"-Wa(??),-mfix-ls2f-kernel". One thing to note is there’s a hunk can’t be patched automatically since the context around it in my gas/config/tc-mips.c file is a little bit different from when r0bertz create the patch. However it’s very easy to manually patch it.
2. Install Bison, Flex and texinfo on my debian-mipsel. To be exactly, I’m not sure if they’re *all* neccessary. But without them, the make process will report error "[bfd.info] error" and "yyparse undeclared indetifier". I google it a little and install these three packages at one time:-)
3. Configure. I use CFLAGS="-O2 -march=mips3 -mabi=n32 -pipe" as Fai told me and "–disable-nls –enable-shared". Of cource the CC="/usr/bin/gcc-4.4".
4. make configure-host && make && make install. The whole process took me 25min on my fuloong2f box.
Things still need to be figured out:
1. There is bin/ diretory in {PREFIX} and another bin/ in mips64el-unknown-linux-gnu. The last one is what i need but what’s the first one for?
2. I’m not sure what "make configure-host" is for.
3. I’m still confused the term "multilib". How many meanings does it have? We can say a multilib system and a mutilib gcc but what’s the relationship between them?
What’s next:
gcc, glibc and kernel.
Summary:
This is probably the easiest package to be built in the tool-chain and it cost me one hard day to learn and try. Though I build it successfully today, many potential errors may come out later when building other packages. However, I still think it as a good start
The first important task when one wants to carry out a historical study of the behavior of a system is to determine what information you want to store and how this is going to be saved, thinking of providing an easy way to access information when needed.
In our case, we need to store the information of the packages status and the actions that have been made to them. For now I have confronted the aptitude state and the dpkg state.
We use the state of dpkg to see the changes that other applications (Apt, Synaptic, Adept, etc..) can perform on the packets.
So the first task was to create routines to save in each member of the history the information that can be collected from pkgstates and the cache, which really is not much. This process is accompanied by the function to write in the text log and the binary database. I believe that work it’s around 70-75%, missing optimization and definitions of some values.
I have been working this week on the aptitude state , creating a vector of entries which is storing the information collected from the state’s changes through the functions cleanup_after_change, apply_solution and build_selection_list from aptitudeDepCache, calling on the last function to the routine that writes the whole vector of changes made so far in the text log and the database, I think this work is located at 50%.
Plan for next weeks
For the dpkg state, optimize the functions and add to the info of each package the date when it changed. Maybe based on /var/log/dpkg.log
For the Aptitude state, finishing collecting the information about changes. For example the dep that trigger a change, groups of packets that share the same reason of change. Look at other relevant information that could be included for the trace of each package. Change the structure of the database to include information that is missing.
by slackydeb (noreply@blogger.com) at June 05, 2009 12:22 PM
by slackydeb (noreply@blogger.com) at June 05, 2009 12:21 PM
by slackydeb (noreply@blogger.com) at June 03, 2009 11:11 PM
by David Wendt (noreply@blogger.com) at June 03, 2009 12:47 AM
Whilst working on update-manager I have been wondering whether I should use gettext for localizing debug output and error messages sent to stderr.
As for debug output itself I basically do not see the need for providing a localized version for each and every message sent to stderr, but as far as error messages are concerned I am uncertain.
The point is that update-manager (apart from its experimental text interface) is usually not launched from a terminal at all and so most users won’t even see these messages ever. Also, I believe that every developer’s English skills are good enough so that he or she is able to understand simple messages.
Error messages however might be useful to all users when they experience a problem with the software, but localizing those could make handling bug reports a bit harder, possibly having to translate the error message back to English before being able to see what has gone wrong.
So basically I am asking you: What do you think? Is it worth localizing these messages? What is your experience with localized or non-localized error and debug messages?
I would be glad if I could get some input from you, either as a comment to this article, via email to debian(dot)sp(dot)or(dot)at or through the update-manager-devel mailing list.
As I promised to keep you updated on recent developments on update-manager I am writing this article. Just as a disclaimer: I am not going to write about any recent developments here, but would rather like to point at a piece of code I added to update-manager that could be useful in other applications too.
Now, as the title suggests there are sphinx-aware Enums in update-manager. Enums are common constructs in other programming languages like C and allow simple creation of constants with, for example, ascending values (first constant has value 0, second has value 1 and so on). Python unfortunately does not include support for Enums itself, but I found it rather easy to write classes that emulate such a construct.
Nothing is new about Enums in Python and there are probably quite a few different implementations out there, but I believe mine is different. The sphinx-aware part means that my implementation automagically updates the docstrings of the created instances and thus allows sphinx’ “autodata” method to include sensible information in generated API documentation.
I could go on writing about and praising my method, but I believe a short example gives you a better idea how my implementation works and what I wanted to achieve with this. Have a look at this page, which is part of update-manager’s new API documentation. You should see rather well-looking documentation of the UpdateManager.Backend.RELOAD_CACHE_STATUS NegativeEnum, the defined constants, their values and some additional information about each value now.
Still, nothing too fancy, HTML documentation generated from docstrings. What makes this special is the code from which it was generated:
RELOAD_CACHE_STATUS = NegativeEnum( BEGIN = "Started reloading package cache", DONE = "Finished reloading package cache")
This not only gives us a RELOAD_CACHE_STATUS enum, along with the RELOAD_CACHE_STATUS.BEGIN and RELOAD_CACHE_STATUS.DONE, but also some documentation, included in RELOAD_CACHE_STATUS‘ docstring, that can be used by sphinx.
You can find the Enum code, which is rather short and should be quite easy to understand, here. I hope you find this code as useful as I do.
by David Wendt (noreply@blogger.com) at May 30, 2009 02:05 AM
by David Wendt (noreply@blogger.com) at May 29, 2009 03:55 AM
As I noted in this weeks update-manager progress update one of my tasks was to create an alioth.debian.org project and get my branches uploaded to Debian.
I did not imagine that alioth admins (hi there, a huge “thank you” goes to you guys) would be this fast with reviewing and accepting the project and enabling bazaar support for me.
Anyways, the project has been accepted and its new home is on alioth. I have also already uploaded both my update-manager branch and python-apt branch to bzr.debian.org
Additionally I have generated the API documentation, which is also hosted on alioth, and created a development disccusion mailing list, update-manager-devel at lists.alioth.debian.org.
If you are interested in this project feel free to have a look at what I’ve done so far and join the development discussion. Comments, critizism and ideas are always welcome.
It has been more than a month since I last wrote about my work on update-manager during this year’s Google Summer Of Code and I am somewhat ashamed I wasn’t able to provide you with updates more regularly.
So first of all, yes, I did do some work and yes, there has been quite some progress. Basically both private and university stuff have kept me from writing and that’s why I’d like to start with this series of weekly updates today.
This series are meant to summarize what has happened during a week of writing code and give you an overview of what’s happening. This first issue however will sum up the past month.
So let me begin explaining what has happened since my last post.
update-manager bazaar repository
All code I have written so far is available through a public bazaar branch on launchpad.net. My branch’s page can be found here and provides you with its history and of course instructions on how to obtain the code. The location is only temporary though, as I am going to move hosting over to alioth.debian.org. This is on my task list for next week.
modular design
I have ripped apart nearly all of update-manager and put it together in a more modular way, which should implementing new frontends or backends more easy, whilst also simplifying code maintenance.
The new design consists of four major parts:
backend implementation
When I started working on update-manager it heavily relied on synaptic and used it to do the dirty-work. However, together with my mentor, mvo, I decided to drop synaptic support and rather concentrate on using python-apt. This means that the only backend implementation right now is a python-apt backend.
The python-apt backend is currently a work in progress, but already includes some basic functionality. Right now it can (re-)load the package cache and package lists and is able to provide a list of packages which are upgradable to the frontend.
Whilst implementing these functions I noticed some shortcomings of python-apt itself, fixed those and got mvo included in his python-apt branch at launchpad.
frontend implementation
I started re-implementing the Gtk frontend as provided through current update-manager and right now it visualizes the package cache reloading process and provides users with a list of upgradable packages. However, that’s pretty much all of the functionality it includes right now, which is why implementing more functions is pretty much on the top of my todo list.
Additionally I have ported the text frontend, as included in Ubuntu, to the new modular system, and this frontend’s code really shows how easy adding a frontend with the new modular design is. This frontend contains the same functionality as the Gtk frontend.
distribution specific code
The core described above does not include any distribution specific code anymore, which is the main focus of this project. The implementations of distribution-specific functionality contains classifiers for update categories for both Debian and Ubuntu, whilst I focused on getting things right with the Debian implementation for now. These classifiers allow the frontend to let the user know which kind of update they are about to install, like a security update, a recommended upgrade or a third-party (unofficial) upgrade.
documentation
As update-manager was poorly (read: hardly at all) documented I started documenting the API using sphinx. However, right now the generated documentation cannot be found anywhere yet. This should change as soon as an alioth project for update-manager has been created.
next week’s tasks
I would also like to provide you with my task list for the coming week. The list, ordered by priority, is:
As you can see this list is rather short. This can mainly be attributed to a few university assignments, and instead of providing a long list of tasks which I probably won’t be able to finish I rather keep the list short and hopefully get things not on this list done too.
While working on my Google Summer of Code project today, I came across a bug that pretty much halted my productivity for the day.
Early on in my project, I decided that working with Unicode is hard, among other things. Since I was restricted to using C, I had to find a way to easily manipulate Unicode stuff, and I came across GLib (I’m not even entirely sure how, I think I just remember other projects using it, and decided to look it up.)
Not only did it have Unicode handling stuff, it also provides a bunch of convenient things like a linked list implementation, memory allocation, etc. All in a way intended to be cross-platform compatible, since this is the thing that’s used to power Gtk.
I’m not entirely sure how it differs from Apache’s Portable Runtime (APR); maybe it’s even a Not Invented Here syndrome. In any case, I, not suffering from that particular affliction, decided to be lazy and re-use existing code.
For some reason, GLib’s g_slice_alloc() call was failing. For those of you that don’t know what this does, it is similar to malloc() in standard C – it allocates a chunk of memory and returns it to you, so that you can make use of dynamic memory allocation, rather than everything just being auto variables. In particular, it means you can be flexible and allocate as much or as little memory as you need.
So I spent the entire day trying to figure out why my program was segfaulting. Looking at the output of gdb (the GNU Debugger), the backtrace showed that it was crashing at the allocation statement. No way, I thought, that test is so well-tested, it must be a problem with the way I’m using it.
I changed the code to use malloc() instead of g_slice_alloc(), and the program started crashing right away, rather than after four or five executions with g_slice_alloc(). Not exactly useful for debugging.
I mentioned my frustration with C on the Debian Perl Packager Group channel, and a friend from the group, Ryan Niebur took a look at the code (accessible via a public repository). After a bit of tinkering, he determined that the problem was that I was using g_slice_alloc instead of g_slice_alloc0, which automatically zeroes memory before returning it.
It stopped the crashing and my program works as intended. I’m still left totally puzzled as to why this bug was happening, and I’m not sure if malloc isn’t supposed to be used with structs, or some other limitation like that.
But thanks the magic of open source and social coding/debugging, the expertise of many contribute to the success of a single project. It’s such a beautiful thing.
Update: There were a lot of questions and comments, mainly relating to the fact that malloc and friends return chunks of memory that may not have been zeroed.
Indeed, this was the first thing I considered, but the line it happened to be crashing on was a line that pretty much just did g_slice_alloc, rather than any of the statements after that.
For those that are curious, all of the code is visible in the public repository.
I do realize that the fixes that have been made are pretty much temporary and that they are probably just masking a bigger problem. However, I’m at a loss for the issue is. Hopefully the magic of open source will work for me again, and one of the many people who have commented will discover the issue.
Posted in Computer Science, Software Engineering Tagged: Algorithms, Best Practices, Code Reuse, Computer Science, Google Summer of Code
When starting with Python or just when you haven’t read PEP 8 you usually fall into some frequent mistakes. Mostly because you take your experience in other languages to Python and your mind gets confused with the new conventions (or something like that).
Some of these errors and their explanation:
Read more about this and other conventions in PEP 8.
Cristian Mauricio Porras Duarte
David Wendt Jr.
Diego Escalante Urrelo
Emilio Pozuelo Monfort
Jonathan Yu
Luca Favatella
Philipp Kern
Sha Liu
Stephan Peijnik
Last updated:
November 21, 2009 04:36 PM
All times are UTC.
Powered by:
