Brainacle

UPDATED: Multiseating with KDE and XBMC Like a boss

2012-10-30T23:03:00Z

A while ago, I wrote a how-to concerning a multiseat setup with xbmc and KDE. Since then, I've done a lot of tweaks to the system, optimized the configurations, made them more modular and DE independent. Also, UDEV RULES!

Make sure to read the old article here

You can skip the part with the configurations, but if you have time, see how the old system was and how it is now.

Again, It's best if I cover my behind so:

Backup all the files you'll be changing before you start changing them. I don't take any responsibility for any damage you might cause.

Lets start configuring!

Configuration

Before you start, make sure you have a fresh distro. 3.2+ kernel, xorg 1.10+, evdev drivers for mouse and keyboard.

Finding out BusId of the GPUs

We use BusId of the GPU to configure the "Device" parts in xorg.conf. The BusID is a string that looks like "PCI:1:0:0". To find out the right BusID, use lspci | egrep "VGA|Display". The output should look something like this:

00:02.0 Display controller: Intel Corporation 2nd Generation Core Processor Family Integrated Graphics Controller (rev 09) 01:00.0 VGA compatible controller: Advanced Micro Devices [AMD] nee ATI Juniper XT [AMD Radeon HD 6000 Series]

The output is pretty much straightforward. You can see which Bus is which GPU. So, based on this, your BusIDs would be PCI:0:2:0 and PCI:1:0:0. Save those strings somewhere, you'll need them later.

Configuring Udev

Here's the fun part :). We write udev rules to tag the inputs, so we can assign them to specific seats.

First, we have to distinguish the different inputs for the seats by some identifier. If your input are different models and/or vendors it's very easy. If not, comment me, and if there's enough comments, I'll try to write up a good Udev tutorial on how to find sys paths, query by advanced parameters etc.

So now, open a terminal and write 'lsusb'. The output should look like this:

Bus 001 Device 002: ID 8087:0024 Intel Corp. Integrated Rate Matching Hub
Bus 003 Device 002: ID 8087:0024 Intel Corp. Integrated Rate Matching Hub
Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
Bus 002 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
Bus 003 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
Bus 004 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub
Bus 001 Device 003: ID 046d:c50e Logitech, Inc. Cordless Mouse Receiver
Bus 003 Device 003: ID 1058:1021 Western Digital Technologies, Inc. Elements 2TB
Bus 003 Device 004: ID 046d:c318 Logitech, Inc. Illuminated Keyboard
Bus 003 Device 005: ID 1532:0016 Razer USA, Ltd DeathAdder Mouse

Notice the ID. Its meaning is VENDOR_ID:PRODUCT_ID. My inputs are 046d:c50e, 046d:c318 and 1532:0016.

Save your IDs.

Next, go to /etc/udev/rules.d and create a script called 99-multiseat.rules.

Next few lines should go in it:

SUBSYSTEM=="input", ENV{ID_INPUT.tags}="input_default"
SUBSYSTEM=="input", ENV{ID_VENDOR_ID}=="046d", ENV{ID_MODEL_ID}=="c50e", ENV{ID_INPUT.tags}="input_right"

The first line tags all inputs with tag "input_default". The second line tags the specific input with Vendor ID 046d and model ID c50e with the tag "input right". Change the values to your vendor and model ID.

Save the script, mark it +x and reboot.

Configuring X

Last how-to used one xorg.conf. Now, we'll use main seat and off-seat configuration files for xorg.conf.

Open /etc/X11/xorg.conf for editing, as root user. If you don't have that file, just create it. Note: Different distros might have different xorg.conf path. Most of the distros use /etc/X11/.

We'll use a simple, barebone xorg configuration, and let X autoconfigure everything else:

Section "ServerFlags"
    # Xorg will otherwise not start if it can't find a mouse to use. Better safe than sorry.
    Option  "AllowMouseOpenFail"    "true"
EndSection

Section "Device"
    Identifier "gfx_left"
    Driver "radeon"
    BusID       "PCI:1:0:0"
    EndSection

    Section "InputClass"
    Identifier "ignore_other_seats"
    MatchTag "input_right"
    Option "Ignore" "yes"
EndSection

That's ALL! By using MatchTag, we match the inputs tagged as "input_right" and we "Ignore" them. Note the BussID in the "Device" section. It should match your primary GPU.

Save the script. This should work independently of the next configuration file.

Open /etc/X11/xorg-xbmc.conf for editing, as root user.

Section "Device"
    Identifier "gfx_right"
    Driver "intel"
    BusID       "PCI:0:2:0"
EndSection

Section "InputClass"
    Identifier "ignore_other_seats"
    Option "Ignore" "yes"
EndSection

Section "InputClass"
    Identifier "input_right"
    MatchTag "input_right"
    Option "Ignore" "no"
EndSection

This config will be used by the off-seat xserver. We point to the graphic card, we tell it to ignore all the inputs except those tagged as "input_right" and we're set here.

Testing the configuration

First, let the main xserver load the xorg.conf. Switch to runlevel 3 and then 5 again (telinit [runlevel]).

When the main seat is up, open up a terminal, and execute this:

/bin/su -- - [your off-seat user] -c "xinit -- /usr/bin/X :2 vt9 -nolisten tcp -config xorg-xbmc.conf -sharevts -novtswitch" </dev/tty9 >/dev/tty9 2>/dev/tty9

It should get the server up.

If you want to daemonize this, open up a init script (either /etc/init.d/xorg-offseat or /etc/rc.d/xorg-offseat depending on distro) and inser the following:

#!/bin/sh

set -e

PATH=/usr/local/bin:/usr/local/sbin:/bin:/usr/bin:/sbin:/usr/sbin
DAEMON=/usr/bin/startx
PIDFILE=/var/run/xorg-offseat.pid
USER="[your offseat user here]"
HOME=~[your offseat user home dir here]
CONFIG="xorg-xmbc.conf"

case "$1" in
  start)
    sleep 5s
    echo -n "Starting X session: startx"
    cd $HOME
    /bin/su -- - $USER -c "xinit -- /usr/bin/X :2 vt9 -nolisten tcp -config $CONFIG -sharevts -novtswitch" </dev/tty9 >/dev/tty9 2>/dev/tty9 & echo $! >$PIDFILE
    #/usr/bin/startx -- :2 vt9 -nolisten tcp -config $CONFIG -sharevts -novtswitch"
    echo "."
  ;;

  restart)
    /etc/rc.d/xorg-offseat stop
    /etc/rc.d/xorg-offseat start
  ;;

  stop)
    echo -n "Stopping X session: startx"
    kill `cat $PIDFILE` || echo -n " not running"
    echo "."
  ;;

  *)
    echo "Usage: /etc/init.d/xorg-offseat {start|stop|restart}"
    exit 1
    ;;
esac

You can start this as you start any other daemons. And stop it as well.

If you don't want to use daemons, you can always rely on your display manager. I used KDM, trieg gdm but a lot has changed and the docs aren't much updated, so I can't be sure if it'll work.

Configuring KDM

Open /usr/share/config/kdm/kdmrc for editing, as root.

In the [General] section, change the ReserveServers and StaticServers as follows:

ReserveServers=:2,:3
StaticServers=:0,:1

At the end, add this snippet:

[X-:0-Core]
ServerArgsLocal=-nolisten tcp -config xorg.conf

[X-:1-Core]
ServerArgsLocal=-nolisten tcp -config xorg-xbmc.conf -sharevts -novtswitch

If you want a user to auto-login on the second seat (in my case, I want the xbmc user to autologin), add this to the file:

[X-:1-Core]
AutoLoginEnable=true
AutoLoginLocked=false
AutoLoginUser=xbmc
ClientLogFile=.xsession-errors

Don't execute custom sessions!. It's bugged. Try the default one and autostart xbmc from there.

Things from the old How-To

Tweaking HDMI audio for XBMC

You need to find the alsa sinks of your system. To do that, use aplay -l. The output should be something like this:

**** List of PLAYBACK Hardware Devices ****
card 0: PCH [HDA Intel PCH], device 0: ALC887-VD Analog [ALC887-VD Analog]
  Subdevices: 1/1
  Subdevice #0: subdevice #0
card 0: PCH [HDA Intel PCH], device 1: ALC887-VD Digital [ALC887-VD Digital]
  Subdevices: 1/1
  Subdevice #0: subdevice #0
card 0: PCH [HDA Intel PCH], device 3: HDMI 0 [HDMI 0]
  Subdevices: 1/1
  Subdevice #0: subdevice #0
card 0: PCH [HDA Intel PCH], device 7: HDMI 1 [HDMI 1]
  Subdevices: 1/1
  Subdevice #0: subdevice #0
card 1: Generic [HD-Audio Generic], device 3: HDMI 0 [HDMI 0]
  Subdevices: 1/1
  Subdevice #0: subdevice #0

Next, find out which sink goes to the right HDMI. Use aplay for that:

aplay -D plughw:[cardId],[deviceId] /usr/share/sounds/alsa/Front_Center.wav

Replace cardId and deviceId from the list of playback hardware devices. When you hear a sound on your TV, you've hit the right device. Just remember the card id and the device id.

Go to XBMC System Settings->Audio output. Choose Audio output device - Custom. Insert plughw:[cardId],[deviceId](e.g. plughw:0,7) in Custom audio device. You're done. XBMC should route audio thru your HDMI.

Tweaking policykit

If the second seat is unable to use removable drives, bluetooth dongles, policykit is to blame.

Add/Edit /etc/polkit-1/localauthority/50-local.d/mseat.pkla:

[allow operations]
Identity=unix-group:plugdev
Action=org.freedesktop.udisks.*;org.blueman.*;org.freedesktop.pulseaudio
ResultAny=yes
ResultActive=yes
ResultInactive=yes

That should be it.

If this setup doesn't work for you, don't give up easily. Ask around forums and irc. These setups can differ largely, based on the hardware and the setup. You can ask in comments as well. I'll try to help out as much as I can.

Post Scriptum

Comment me with your issues, I'll try to help as much as I can. Try to leave verbose outputs please. Best of luck with your multi-seat setups!

Multiseating with KDE and XBMC Like a boss

2012-05-27T21:03:00Z

UPDATED with udev, modular configuration and less headache here.

Linux multi-seat setup is a setup where multiple users can use the same PC simultaneously. You have a monitor and input devices for each of those users, and a separate X session for all. It's a PITA to set up if you don't know what to do, but the end result is legendary. Trust me, I spent countless hours setting up mine, but it was worth it.

Why?

You have a spare hardware, and you want to utilize it
You have 1 PC, but you want your sibling/SO/family member to have it's own independent environment
You want to add a TV or multimedia session monitor to your setup that will be independent of your current desktop session (we'll focus on this)
It's awesome!

What do I need?

First off, you need a PC with at least 2 graphic cards. Any combination is good. N discrete GPUs or 1 integrated + N discrete. If you mix graphic card vendors, try to use the FOSS drivers. I haven't tested multiseating with mixed binary drivers or blobs and FOSS drivers. I'll update the post once I test this.

You need at least 1 monitor per seat, or you can switch one monitor back and forward. In the guide (my setup), I use 3 monitors. One dual-head seat and one single-monitor seat. At least one monitor per GPU.

You'll need separate inputs for each seat. They are optional, but how would that work? Try to, at least, get another mouse.

You'll need advanced knowledge on Linux. Sorry, but this is an advanced topic, so If you don't know how to set up xorg.conf, you are in for a lot of frustrations.

Iron will and nerves of steel: It might be a bumpy ride, so brace yourself.

My scenario

I recently purchased a TV and wanted to hook it up to the PC, pop XBMC on it, and, if possible, make it work with separate user/input/session. A perfect use-case for a multi-seat. I already had a dual-head setup with an ATI Radeon HD6770. My PC has an integrated HD2000 GPU (i3-2120) which I could use for the second seat. Pics:

2 monitors on the Ati, TV tied to the onboard graphic card via HDMI

Main seat - dual-head, KDE session

XBMC running on the TV in separate session/seat

KDE running on the TV in separate session/seat

I could have gone with the triple-head setup, but:

It puts a lot of strain to the main GPU
Vsync is out of whack
It bugs out often
All my monitors are different sizes/resolutions.
Running XBMC on fullscreen on the third monitor is a pain.

With multiseat, I run xbmc automatically, fullscreen, in a separate session and a separate user.

My distro is Archlinux. The configurations or config file paths might differ in other distros, but it shouldn't be a problem.

Backup all the files you'll be changing before you start changing them.

Lets start configuring!

Xorg Configuration

Before you start, make sure you have a fresh distro. 3.2+ kernel, xorg 1.10+, evdev drivers for mouse and keyboard.

Finding out BusId of the GPUs

00:02.0 Display controller: Intel Corporation 2nd Generation Core Processor Family Integrated Graphics Controller (rev 09) 01:00.0 VGA compatible controller: Advanced Micro Devices [AMD] nee ATI Juniper XT [AMD Radeon HD 6000 Series]

Finding out Device Path of the Input devices

Plug in all the inputs you'll be using. Use ls /dev/input/by-id. The output should look like this:

usb-Logitech_Logitech_Illuminated_Keyboard-event-if01 usb-Logitech_Logitech_Illuminated_Keyboard-event-kbd usb-Logitech_USB_RECEIVER-event-mouse usb-Razer_Razer_DeathAdder-event-mouse

From this list, you can identify your inputs. If you have same vendor/model inputs, you can reference them by bus and device ID, and even by the simplest form /dev/input/eventX. I suggest you use some specific referencing - by id or by device id.

So, my device paths are:

/dev/input/by-id/usb-Logitech_Logitech_Illuminated_Keyboard-event-kbd
/dev/input/by-id/usb-Logitech_USB_RECEIVER-event-mouse
/dev/input/by-id/usb-Razer_Razer_DeathAdder-event-mouse

That's all we need for now. Let's pull all of this in xorg.conf:

Configuring X

This is the most important step in making a multiseat. We work solely on xorg.conf.

Open /etc/X11/xorg.conf for editing, as root user. If you don't have that file, just create it. Note: Different distros might have different xorg.conf path. Most of the distros use /etc/X11/.

We'll use a simple, barebone xorg configuration, and let X autoconfigure everything else:

Section "ServerFlags"
    # Xorg will otherwise not start if it can't find a mouse to use. Better safe than sorry.
    Option  "AllowMouseOpenFail"    "true"
EndSection

Section "InputDevice"
        #Configuring the keyboard for the main seat
        Identifier "keyboard0"
        Driver "evdev"
        Option "Device" "/dev/input/by-id/usb-Logitech_Logitech_Illuminated_Keyboard-event-kbd"
EndSection

Section "InputDevice"
        #Configuring the mouse for the main seat
        Identifier "mouse0"
        Driver "evdev"
        Option "Protocol" "Auto"
        Option "Device" "/dev/input/by-id/usb-Razer_Razer_DeathAdder-event-mouse"
EndSection

Section "InputDevice"
        #Configuring the mouse for the second seat
        Identifier "mouse1"
        Driver "evdev"
        Option "Protocol" "Auto"
        Option "Device" "/dev/input/by-id/usb-Logitech_USB_RECEIVER-event-mouse"
EndSection

Section "Device"
        #Configuring the discrete GPU
        Identifier "radeon"
        Driver "radeon" #Or a driver specific to your GPU - "nvidia"
        BusId "PCI:1:0:0" #From the BusID section above
EndSection

Section "Device"
        Identifier "intel"
        Driver "intel" #Or a driver specific to your GPU - "nvidia", "radeon"
        BusId "PCI:0:2:0" #From the BusID section above
EndSection

Section "Screen"
        # Configuring the screen for the main seat
        Identifier "screen0"
        Device "radeon"
EndSection

Section "Screen"
        # Configuring the screen for the second seat
        Identifier "screen1"
        Device "intel"
EndSection

Section "ServerLayout"
        #Configuring the Server layout for the main seat
        Identifier      "main"
        Screen          "screen0"       0                   0
        InputDevice     "mouse0"        "CorePointer"
        InputDevice     "keyboard0"     "CoreKeyboard"
        #This is a must - otherwise, it will add all the inputs to this seat
        Option          "AutoAddDevices"        "off"
EndSection

Section "ServerLayout"
        #Configuring the Server layout for the second seat
        Identifier      "tv"
        Screen          "screen1"       0                   0
        InputDevice     "mouse1"        "CorePointer"

        #If you have a spare keyboard, make another InputDevice entry for it and add it here
        #InputDevice     "keyboard1"     "CoreKeyboard"

        #This is a must - otherwise, it will add all the inputs to this seat
        Option          "AutoAddDevices"        "off"
EndSection

Next, we configure the session manager

Configuring KDM

Since I use KDE, I'll show you how to configure KDM. If you use other DE, please google around, it's mostly similar to this.

Open /usr/share/config/kdm/kdmrc for editing, as root.

In the [General] section, change the ReserveServers and StaticServers as follows:

ReserveServers=:2,:3
StaticServers=:0,:1

At the end, add this snippet:

[X-:0-Core]
ServerArgsLocal=-nolisten tcp -layout main

[X-:1-Core]
ServerArgsLocal=-nolisten tcp -layout tv -sharevts -novtswitch

If you want a user to auto-login on the second seat (in my case, I want the xbmc user to autologin), add this to the file:

[X-:1-Core]
AutoLoginEnable=true
AutoLoginLocked=false
AutoLoginUser=xbmc
ClientLogFile=.xsession-errors

And, if you want a custom session to be executed, not the default - KDE, you'll need 2 things:

A shell script that would execute the session (exec xbmc-standalone), marked executable
Add a line in [X-:1-Core] - Session=/path/to/that/shell/script

That's it for KDM. If you followed and saved everything, you can restart X now. You don't need to reboot (but do it anyway). Next login, you should see different sessions on different GPUs

If you have any problems starting the X server, just move or remove xorg.conf, and revert kdmrc. Keep backups of the old files, so you can revert fast

Fine tuning

Now, I have a KDE session on my main seat, and xbmc session on my second seat. But no sound... Making the sound work on a multi-seat system is quite difficult and it differs from the type of sound architecture.

In my scenario, I feed my audio through the HDMI cable to the TV, and it's quite easy to do so. I have pulseaudio on my system.

Tweaking HDMI audio for XBMC

You need to find the alsa sinks of your system. To do that, use aplay -l. The output should be something like this:

**** List of PLAYBACK Hardware Devices ****
card 0: PCH [HDA Intel PCH], device 0: ALC887-VD Analog [ALC887-VD Analog]
  Subdevices: 1/1
  Subdevice #0: subdevice #0
card 0: PCH [HDA Intel PCH], device 1: ALC887-VD Digital [ALC887-VD Digital]
  Subdevices: 1/1
  Subdevice #0: subdevice #0
card 0: PCH [HDA Intel PCH], device 3: HDMI 0 [HDMI 0]
  Subdevices: 1/1
  Subdevice #0: subdevice #0
card 0: PCH [HDA Intel PCH], device 7: HDMI 1 [HDMI 1]
  Subdevices: 1/1
  Subdevice #0: subdevice #0
card 1: Generic [HD-Audio Generic], device 3: HDMI 0 [HDMI 0]
  Subdevices: 1/1
  Subdevice #0: subdevice #0

Next, find out which sink goes to the right HDMI. Use aplay for that:

aplay -D plughw:[cardId],[deviceId] /usr/share/sounds/alsa/Front_Center.wav

Replace cardId and deviceId from the list of playback hardware devices. When you hear a sound on your TV, you've hit the right device. Just remember the card id and the device id.

Tweaking policykit

If the second seat is unable to use removable drives, bluetooth dongles, policykit is to blame.

Add/Edit /etc/polkit-1/localauthority/50-local.d/mseat.pkla:

[allow operations]
Identity=unix-group:plugdev
Action=org.freedesktop.udisks.*;org.blueman.*;org.freedesktop.pulseaudio
ResultAny=yes
ResultActive=yes
ResultInactive=yes

That should be it.

Post Scriptum

The "howto" is quite big for me to follow. If you feel I've missed anything, notify me in comments and I'll update the post. Thanks for understanding.

I want to thank the Archlinux and Gentoo communities for their effort on the wikis. They helped me a lot in making this. Also, kudos to the whole Linux community for doing things this awesome.

And, XBMC devs, hats off to you guys!

KDE vs. Gnome - A Subjective View : UPDATE

2012-04-12T00:03:00Z

Few months ago, I wrote an article on my view on the KDE vs. Gnome - Pros, Cons, stuff that irritates me and stuff that's good. At that time, I was still a Gnome user. But after a lot of frustrations with the Gnome desktop, I switched to KDE and never went back to this date. After KDE SC 4.8 dropped in. Archlinux immediately picked it up, as always, so I had a lot of time to test drive it and see how it preforms.

This update is about my view on the updated KDE 4.8 and everything I did to make my KDE experience better (things that were on the cons list last time). As a bonus, a bit of ranting about the current state of Linux on the Desktop.

Testing Ground

Another thing that changed in the meantime, was my workstation:

i3-2120 3.3GHz

Asus P8Z68-V LX

8GB ddr3 ram

Ati Radeon HD6770 1GB DDR5 with fglrx (binary driver AND radeon) + NVidia GT440 + HD2000 - tested on all cards

1x22"@1680x1050 + 1x24"@1920x1200 monitors

Archlinux, latest updates and some Sabayon KDE tests

Enter KDE 4.8

On first looks, KDE 4.8 is not too much different from 4.7. The panel is reworked a bit, so it looks slicker, dolphin does tricks when resizing, tweak here and there. The real update KDE 4.8 brings is increased performance. It's truly faster. For the first time, OpenGL renderer is faster than XRender, even with VSync on. Woot! Now, I can enable desktop effects and don't worry about performance drop. On all cards and all drivers!

This might not strike you as an update, but at this moment, it's joy to me - KDE still keeps the original desktop metaphor as primary, and adds on that an even better optional interface, optimized for small/touch displays. I bolded optional because everybody seems trigger happy when dealing with touch displays, destroying the good ole' desktop in the process (yes. I'm looking at you Gnome. You too Ubuntu!). Anyway, for the interested, check out Plasma Active.

Dolphin

Still no mouse back-button. A bit bummer, but i learned to live with it. At least, dolphin has better UI than any GTK based file manager, so going back and forward, up and down, is easy with mouse/keyboard. You could try the hack with xbindkeys, but apparently, it hates my Razer Deathadder.

KIO

Nothing has changed here. And I doubt anything will, so, I did a few tweaks - I'm using smb4k for smb hosts, and VLC has implemented KIO slaves! I can live with this and so can you.

KDE-Wallet

I was under the impression that KSecretService will change things, but it wasn't what I was hoping for. So, I made a bit of tweaking here too. I use keychain and add eval `keychain --eval --agents ssh -Q --quiet id_dsa` to my .zshrc/.bashrc. Then, if you use openssh ask pass, KDE wallet can save the password. Works like a charm. Eat that gnome keyring!

General / Performance bugs

A lot has changed here. There are no corruptions (to date) in the plasmoids, panels etc. All works good. Resizing dolphin is fast (except with some NVidias+blobs). There was a significant lag when one starts dragging window that's not focused. It works great now.

The default application bundle is still better than Gnome's (seriously, compare Evince to Okular, Brasero to K3b, Gwenview to any gnome image viewer).

It still looks frickin' GOOD! It's so good looking and configurable, Gnome looks like an OS intended for some child-centric laptop. You can tweak every little bit of it. And they should start selling this point! Average new user won't be aware of this and might revert him from KDE. Seriously guys, Moar marketing nao!

There were some screenshots of KDE made to look like Unity/Gnome3. If you want that look, you can have it .

And the most important, it didn't betray the users that respect and like the desktop metaphor, just so it can be in the competition for the best UI for the tablets/phones/TV. It still is - with Plasma Active, and by the looks of it, it already scores big time, but they stayed loyal to the old ways. The way a desktop is meant to be.

Gnome 3 / Unity

Sadly, nothing has changed here. They really stuck with it. Good for you guys! One thing tho. When a user needs 30 different apps to change the UI, you aren't doing a good work. I've been a PC and Linux user for many years, I have friends with the same/similar experience, and neither one of them has said good things about G3/Unity. Few friends were so appalled by the current state, they switched to Mac after all these years (KDE was not an option because of the issues mentioned in the previous post).

The only "serious" DE that promises, is Cinamon by Linux Mint dudes, but:

1: It's a long shot. The community working on it is small and poorly financed

2: It's new. XFCE has been on the scene longer, yet it can't come even remotely close to the usage of Gnome/KDE

3: It's Gnome3 at it's core!

And mate...

Bonus: My view on the whole Desktop thing

After a long time of me ranting about the slug that was KDE compared to Gnome, I caved in and moved (for good i hope). I owe some people apologies. KDE community is focused on building and improving stuff. Not tearing the old things and starting from scratch. I know, I know. KDE 3 to 4. Now, think about how big of a change was that, and how much Gnome 2 changed when it hit 3. We all owe KDE apologies!

As a bonus, here are some of my thoughts on Linux on the Desktop:

No, it's not the year of Linux for the Desktop. It's (and will be) the year of Linux for the smartphone. If we want to live to see Linux on the desktop, PEOPLE SHOULD USE IT!. A lot of people say they use Linux, but they don't. Either they use it in VM or dual boot it with Windows as a secondary - testing machine.

In order for people to use Linux on the desktop, a lot of things have to change.

First and foremost, stop the effin' fragmentation! 2 DEs were enough. 4 are a lot. 6 are shaving yaks. KDE is doing the right thing at the moment. Switch to it. You'll thank me later. That will make some of the popular - entry-level distros to support their KDE flavors more and we'll stop bitching about the damned tablet mentality.

Second: Great Unified Package manager. See Mac's bundles. People have big HDDs now, you can ship the libs in the same folder even if the lib is present in the system.

Third: Stop focusing on minor things and fix the Elephant. Ubuntu folk were so excited about the new test printer template, but I can't see it because I can't make my printer work on Linux!

Fourth: For Buddha's sake, stop the support for antics. There's enough backlog of every distribution in history to power anything from Eniac to a super modern workstation. Slim down on support for old hardware.

And most important: Stop being purists and elitists. So what if a distro ships with non-free drivers? Boo-hoo! Are you trying to chase even those that stick to Linux? Personally, I've been FOSS supporter for a lot of time, but even I don't want to limit myself because there is no (truly) free support for my hardware. NVidia/Ati blobs are free of charge - aren't they? So what if a user uses DE instead of Awesome/ratpoison/dwm/xmonad? Not all Linux users are sysadmins or Haskell hackers, you know. Don't call them newbs. Focus a bit more to the user. Ubuntu did that, and was reign of the Linux on the desktop until the Great Tabletation. They had the chance, and they blew it.

Believe me, I want to see Linux in the top percentages of usage as much as the next guy, but in order for that to happen, I have to make my wife, my brother, my aunt, my cousins to use Linux. It's a tough task. They can't install apps, they can't install Linux on their cheap laptops, they can't make their wi-fi/printer work, they can't find the start button! And they make for the majority of users! Tech savvy people are small percentage, and they tend to focus on the work instead hacking config files, compiling mesa stacks, chasing windows and hot spots for activating the launchers. Make it work OOB. Let's copy Windows for a change :) (just kidding... or am I).

P.S. -again- For the purists s/Linux/Gnu\/Linux/

How to use Django Uni-form with Twitter Bootstrap CSS

2012-02-13T20:58:00Z

I'm sure, most of you, know what django-uni-form is... For those that don't, django-uni-form is a very cool django app that makes form management/styling/layouting a breeze. You can do pretty much anything to a form with it: Split it in formsets, add HTML between every element, wrap different elements in different containers etc. Also, it produces great HTML, and it manages the <form> tag, as well as the submit/reset buttons. If you aren't using it, you should definitely start.

Twitter Bootstrap is a client side framework by twitter, encompassing many aspects - layout, sane default CSS, typography, Javascript plugins, even some icons. It looks very professional, it's easy to use and very popular. The way bootstrap styles a form is what most people need when building or prototyping a web app. But, you need to be able to change some css classes and html elements in order to work.

Doing this with django-uni-form is extremely easy. All we need to do is change the default field template. I'll start with an example:

Bootstrap

This step is easy. Just make sure you have downloaded the Bootstrap CSS file, and that it's properly loaded.

The HTML

Simple HTML is all we need:

{% load uni_form_tags %}
<!doctype html>
<html>
<head>
  <title>Django Uniform and Bootstrap</title>
  <!-- LOAD BOOTSTRAP CSS HERE -->
</head>
<body>
  <header class="navbar">
  </header>
  <div role="main" class="container">
    <div class="row">
        <div class="span12">
            {% uni_form form form.helper %}
        </div>
    </div>
    <hr/>
    <footer>
      Brainacle 2012
    </footer>
  </div>
  <!-- LOAD BOOTSTRAP JS HERE -->
</body>
</html>

The Form

I'm making a simple User edit ModelForm, with one extra text field:

from django import forms
from django.contrib.auth.models import User
from django.core.urlresolvers import reverse

from uni_form.helper import FormHelper
from uni_form.layout import Layout, ButtonHolder, Submit, Fieldset


class UserEditForm(forms.ModelForm):
    bio = forms.CharField(label = _('Bio'), required = False,
                    widget=forms.Textarea(attrs={'class':'span12'}))

    class Meta:
        model = User
        fields = ("username", "first_name", "last_name", "email")

    def __init__(self, *args, **kwargs):

        self.helper = FormHelper()
        self.helper.form_method = 'POST'
        self.helper.form_action = reverse("user_edit_view_name")
        self.helper.form_class = 'form-horizontal'

        self.helper.layout = Layout(
            Fieldset(
                'Edit Account Info',
                'username', 'first_name', 'last_name',
                'email', 'bio', 'save',
            ),
            ButtonHolder(
                Submit('save', 'Save', css_class='btn btn-large btn-primary pull-right')
            )
        )
        return super(UserEditForm, self).__init__(*args, **kwargs)

Notes: We add span10 class to the "bio" field textarea because labels will take 2 spans, and we want the textarea fill the entire width. We add class "form-horizontal" to the form, to get the labels position in line with the inputs. The button classes are also important.

The Magic

Now, we have to tell uni-form to generate html that will be compatible with Bootstrap. Lucky enough, uni-form provides us with the ability to override the generic field HTML template. Just make a new file called field.html inside [template root path]/uni_form/ folder with the following content:

{% if field.is_hidden %}
    {{ field }}
{% else %}
<div class="control-group {% if field.errors %}error{% endif %}">
    <label for="{{ field.auto_id }}" class="control-label">
        {% if field.field.required %}<b>{% endif %}{{ field.label|safe }}{% if field.field.required %}*</b>{% endif %}
    </label>
    <div class="controls">
        {{ field }}
        {% if field.errors %}
            <span class="help-inline">{% for error in field.errors %}{{ error }}<br/> {% endfor %}</span>
        {% endif %}
        {% if field.help_text%}
            <p class="help-block">
                {{ field.help_text|safe }}
            </p>
        {% endif %}
    </div>
</div>
{% endif %}

This template handles labels, required fields, help text, errors etc. The end result looks something like this:

New Year's Python Meme 2012

2011-12-28T14:01:00Z

Following Tarek Ziade's #2012pythonmeme:

1. What’s the coolest Python application, framework or library you have discovered in 2011?

gunicorn - We are on our way to change every apache installation to gunicorn. Proved to be just as good and quite possibly, better.

haystack - I've built quite a few apps that employed solr, and every time I did that, I wrote the solr config and client API by hand. I was a bit skeptic about Haystack, but now that I've tried it in production, i won't be writing any more solr client APIs any more.

python opencv - Still haven't used it in production, but it's a very cool library.

2. What new programming technique did you learn in 2011 ?

Meta programming in Python. I worked on a project that relied heavily on meta programming. It helped me get more insight in django's models and magic.

3. What’s the name of the open source project you contributed the most in 2011 ? What did you do ?

Around late November and throughout December, I started contributing to AskBot. Here is my contribution so far. 2011 was a very busy year for me, didn't have much time to contribute. I hope things will change to better next year.

4. What was the Python blog or website you read the most in 2011 ?

/r/python. Every important news in the python community lands there pretty fast, and the discussions can be both entertaining and educational.

5. What are the three top things you want to learn in 2012 ?

1. Qt / PyQT - IMHO Qt is the desktop toolkit that evolves the best of all. It has a great community, good documentation and a very rich feature set. It's cross-platform, and even mobile platforms (Android/iOS) are starting to support it.

C - I slowly started brushing up my C skills during this month, and I want to learn more advanced topics in C next year.

3. Pypy - Haven't had the time to test this bad boy better. I have played with it couple of times, but I want to make a serious effort and see the advantages it brings.

If it's not too much already, I want to learn Scheme.

6. What are the top software, app or lib you wish someone would write in 2012 ?

Python based Redmine... Better python based redmine.

Python based reporting software. Something like JasperReports

Out of the context, but I wish i can switch to Python3. I wish more libs get ported since there are some very good features of py3 i want to exploit.

Want to do your own list? Here’s how:

Copy-paste the questions and answer to them in your blog

Tweet it with the #2012pythonmeme hashtag

KDE vs. Gnome - A Subjective View

2011-10-29T21:03:00Z

Past year, a lot has changed with the desktop environments for Linux. Gnome going 3, Ubuntu going Unity, people going nuts about that. I've been experimenting with all of them trying to find the one that will serve me good and will evolve at a slower pace. I, being an exclusive computer/Linux user for a long time, had tough time with the new UIs (I prefer the old style - 1 panel on the bottom, no wobbly-wobbly). My distro of choice is Arch, and my DE of choice was Gnome + Openbox as a window manager. That combo proved to be fast, usable, considerate to hardware yet elegant and (at this moment of time) old-school. I should disclose that I use my primary desktop for everything, not just work (multimedia, internet, gaming) and as such, tilling WMs are a no-no.

Then, Gnome turned 3 and everything changed. Openbox was no longer an option (it has issues even on fall-back mode) so I was stuck with either metacity or mutter. At that moment I started looking into the alternatives. Spent some time on KDE, got LXDE on my netbook (and tried it on my desktop), even tried XFCE for a couple of weeks. This is my subjective view on why I stuck to Gnome 3 fall-back, what I hate about it, what I love and hate about KDE and why I can't work on KDE.

Testing Ground

Before I start, I have to specify my testing ground. You might think that it's irrelevant, but I assure you, hardware makes big difference. Also, graphic drivers matter too. So, my spec:

C2D E4600 2.8GHz (2.4 stock)

Asus P5P43TD-Pro

8GB ddr3 ram

NVidia GT440 OC-ed with Nvidia blob (binary driver)

1x22"@1680x1050 + 1x24"@1920x1200 monitors

Archlinux, latest updates

One of the best things about Arch is that you get almost vanilla versions of the DE in question. Kubuntu's KDE is tainted with additions and configurations, and won't be fair to be judged like that.

KDE

Look at this beauty! KDE is my secret love. It's that good. It looks clean and good, it's the old way (one panel at bottom, desktop is working area), it is feature packed and it evolves possibly the best of all of the DEs. And I would gladly use it! I will install it and not look back right this moment, if it weren't for some annoying stuff that make me rage every time I work on KDE:

Dolphin

No mouse back-button?! What's up with this! It's the 21st century. This has been implemented in almost every graphical file explorer since '98. And yeah... I know about the xautomation and xbindkeys fix, but I want this implemented out of the box. I really don't think it's hard to be implemented.

KIO

So this is the KDE version of gvfs. NOT! It's not even implemented in every general-use software. My pain with this? If I want to open a movie from a samba share with VLC, I have to wait for it to be temporarily downloaded to my PC and then watch it. On gnome, I just open it, and watch it. The even funnier part is, if I use Kate in gnome, it automatically works with gvfs. Why don't you just integrate gvfs or make KIO work similarly?

KDE-Wallet

I don't like to input password every time I do a git pull/push, sudo etc. The KDE-wallet handles ssh auth poorly. There is a fix for it to work with ssh-agent, but why should I make fixes and patches when Gnome's keyring works out of the box?

General / Performance bugs

IMO, my configuration is pretty decent to run the DE on full power. Well, apparently, KDE wants a dual-SLI 580GTX. If I try to turn on the effects, KDE is unusable. If I turn them off, the interface shows random artifacts on screen. And this doesn't happen just on NV hardware. ATI story is worse! Same with sandybridge graphics.

Often, it happens, when I click on the start menu, the menu starts at the top of the screen. Or the panel is positioned on the top half. When I try to reposition it, it slowly falls back to the original place. Happens every restart.

That being said, KDE handles multiple monitors best of all DEs. I have to give credit for that.

All in all, KDE is pretty decent DE. If they fix this few things, I will move back to it immediately and use it for good.

Gnome 3

My current setup. Fallback mode. Never really settled for shell. It's hard for a man working years and years under one desktop model, to switch to something new. I'm fairly satisfied with Gnome3.2 fallback, but, i'm here to bash the things I don't like:

Looks

Who thought that having a solid black bar for a panel was a good idea? Tint2 has better design out of the box. And, I was under the impression that Gnome had some good designers in their team. Well, compare Faenza icons to default gnome ones, and Adwaita to Zukitwo theme. You can do better than this Gnome. We care about the design too.

Gnome settings

Take a look at KDE's settings manager. Now, back to yours. KDE is superior in this matter and you should learn from it. I assume it's laziness that made you throw everything in a registry-like structure (gconf/dconf), but you could move some important stuff in the settings manager. Now, if I want to change your mediocre theme for something better, I have to install third party software for that. Or try to remember Was it dconf or gconf?; Was it desktop/ or apps/. Seriously, work on this. And don't even get me started on the screen-saver thing.

Plain Wrong stuff

Mutter in shell, metacity in fall-back. Did you stop to think that some people would want snap-to-edges in fall-back mode too? Do you punish us, fall-back users, and stick us with metacity? Metacity has an awful compositor. Out of the box, when minimizing the windows, you see this ugly black border. So, the fix is, put metacity in low resources. How is rendering a full window minimizing vs. rendering a black border use lower resources? If you, however, turn that on, you see a wireframe each time you drag a window. So you gotta turn on some other thing (I can't remember which was it and I never can!). It's not documented properly, and I have to google it every time I install a new machine.

When I assign a primary monitor, that means I want that monitor to be PRIMARY! Not the one on the LEFT! KDE handles this better. Learn from it.

QT apps looked good in Gnome. Now, the look good, but they don't work good. Menus flying all around.

Theme manager.

Bundle bash

Brasero sucks hard. K3b is much much better. If you don't trust me, I will show you my bag full of failed DVDs.

Evince is good. But please... Bookmarks. A bit more functionality. You are trying to push Linux on a normal user.

Seriously... Theme manager.

Shell

Oh, my "favorite". I gotta say this - it works better than Unity. Not good enough.

It handles dual monitor poorly. Each time I open window preview, If I have a window on the other monitor, it's also shown in a preview.
Notifications are far from noticeable - fallback mode handles notifications better.
If you have Nautilus managing the desktop, you are in for headaches.
Again, black bar?!
How can I access Home folder without opening window preview?
Constant crashes!
Addons become incompatible fast.
Ati blob issues.
Wrong crash management. Shell crashes, I don't want to restart GDM. Show me ANYTHING. Just let me hit save before restarting X.
And the worst part - Recent files in menu. When you open up the menu, If you don't magically disable (there's a hack for this) recent files, shell becomes awfully slow. Give me a checkbox where I can disable them.
THEME MANAGER!

With all this said, Gnome has less impact on productivity, and I can live with these issues. But as soon as KDE fixes it's quirks, I'm moving to it.

Conclusion

Now, don't focus on the negative stuff I wrote above. It's subjective. Both DE's are good and both flawed. For me, there's a handful of issues that keep me from moving to KDE, and I really hope, they will be fixed soon.

I salute the community for making things like KDE and Gnome. They rock and they should be praised for that. But, if we want Linux to be a choice of the normal (non-tech-savvy) user, you gotta get some things straight. Not everyone knows how to configure using gconf/dconf, or configure xautomation/xbindkeys. Make it perfect.

What ticks you about this Desktop Environments?

P.S. For the purists s/Linux/Gnu\/Linux/

HOW-TO: Use multiple WMD editors on a same page

2011-10-27T15:30:00Z

WMD editor resembles Stackoverflow's Markdown editor and it's probably one of the best WYSIWYG editors that work with Markdown as input. Because of that, it's widely used in Stackoverflow clones (i.e. OSQA). It's easy to install and use (just tie it to a textarea), it supports the mostly used features, provided by this type of editors, but has 1 major flaw: It doesn't support multiple textareas/multiple editors per page.

I've been working on an OSQA based platform recently when I stumbled across this issue. Since we didn't want to change the editor completely, I had to improvise.

Javascript to the rescue!

Going through WMD's code, I found a way to re-initialize the editor with different textarea. When that goes through, the tool bar will be bound to the newly marked textarea and it will edit it's content. The fix is pretty simple actually (my example uses jQuery, but i'm sure you will understand what it does):

    $(".wmd-textarea").focus(function(){
            // Execute this when a textarea with a class .wmd-textarea gains focus

    $("#wmd-button-bar").prependTo($(this).parent("div")); // Prepend the toolbox to the parent div of the textarea

    Attacklab.panels.input = this; // Set WMD's textarea input to this textarea
    Attacklab.editor(); // Re-init the editor.
});

What this block does is hook a function to the focus event of the textarea. Each time the textarea gains focus, it prepends the toolbox to the input's parent element, sets it as the input and reinitializes the WMD editor. That simple.

If you don't want the toolbox to move -- i.e. you want a toolbox above each textarea, just paste the toolbox HTML as desired, point Attacklab.panels.buttonBar to the toolbox tied to the textarea with the focus on it, and re-init the editor:

    $(".wmd-textarea").focus(function(){
            // Execute this when a textarea with a class .wmd-textarea gains focus

    $("#wmd-button-bar").prependTo($(this).parent("div")); // Prepend the toolbox to the parent div of the textarea
    Attacklab.panels.buttonBar = $(this).parent("div").find(".wmd-toolbox"); // Set WMD's textarea toolbox
    Attacklab.panels.input = this; // Set WMD's textarea input to this textarea
    Attacklab.editor(); // Re-init the editor.
});

Note The above code blocks would work with this markup:

<!-- HAVE A WMD EDITOR INITIALIZED SOMEWHERE IN YOUR HTML -->
<div>
        <div class="wmd-toolbox"><!-- HERE GOES TOOLBOX HTML - ONLY FOR PRESENTATION - REMOVE ID'S. COPY IT FROM YOUR FULLY RENDERED PAGE SOURCE. --></div>
        <textarea class=".wmd-textarea"></textarea>
</div>
<div>
        <div class="wmd-toolbox"><!-- HERE GOES TOOLBOX HTML - ONLY FOR PRESENTATION - REMOVE ID'S. COPY IT FROM YOUR FULLY RENDERED PAGE SOURCE. --></div>
        <textarea class=".wmd-textarea"></textarea>
</div>

Nautilus-minus extension

2011-02-14T19:01:00Z

nautilus-min.us is an extension for nautilus that makes uploading images and galleries to http://min.us easier for the user. It adds an item in the context menu, so when you select one or more images, you click "Upload to min.us", wait for a moment, and your default browser will open on the gallery page.

Installation

Automatic installation

You can install the extension from this deb file. It should take care of all dependencies etc. Remember to restart nautilus after installation:

nautilus -q or killall nautilus

If this method fails, try manual installation:

Manual installation

Get the tarball with the latest source here, Extract it into $HOME/.nautilus/python-extensions/ . If there isn't such directory, create it with your user. Extract the tarball so that the script upload_to_minus.py and the dir minus_utils are directly in python-extensions/ and not subfolders.

The extension needs python-nautilus package. On Ubuntus or Debians, install it using apt:

sudo apt-get install python-nautilus

The extension notifies the user via libnotify (pynotify) which is present on Ubuntus and some Debians. If that's not there, it falls back to zenity, which is in standard gnome installation. Try to get at least one of these packages so you can be notified of eventual errors/ updates.

Future

I would love to hear some comments, critique, feature requests etc. You can fork it, request pulls and comment on GitHub.

How to write vim plugins with python

2011-02-03T14:58:00Z

I'm not going to dive into how good or extendible Vim is. If you are reading this article, you probably know that. The thing that makes Vim so good, is the scripting environment behind it called VimL. Using this scripting language, you can write any functionality/plugin you need for Vim. Each plugin you use is written in this language. Here's the best part. You only need very little knowledge of VimL to be able to write plugins, if you know Python (or Ruby).

What's a vim plugin anyway

A Vim plugin is a .vim script that defines functions, mappings, syntax rules, commands that may, or may not, manipulate the windows, buffers, lines. It is a complete piece of code with some specific functionality. Usually, a plugin consists of several functions mappings command definitions and event hooks. When writing vim plugins with Python, often, everything outside the functions is written in VimL. But those are vim commands and they can be learned fast. In fact, VimL can be learned fast, but using python gives so much flexibility. Think about using urllib/httplib/simplejson for accessing some web service that helps editing in Vim. This is why most of the plugins that work with web services are usually done in VimL+Python.

Any prerequisites?

You must have vim compiled with +python support. You can check that using the command:

vim --version | grep +python

Vim package in Ubuntu and it's derivatives comes with +python support.

To Work - Vimmit.vim

What's better than starting with a simple example? This is a plugin that, when called, will retrieve the homepage of Reddit and will display it in the current buffer.

Start by opening "vimmit.vim" file (in vim). Since we are writing python code, its good to check if Vim supports Python:

if !has('python')
    echo "Error: Required vim compiled with +python"
    finish
endif

This piece is writen in VimL. It's best if we stick to VimL for things like this, mappings and event hooks. This function will check if Vim has python support or it will end the script with an error message.

We continue with the main function Reddit(). This is where we use Python and do the main functionality:

" Vim comments start with a double quote.
" Function definition is VimL. We can mix VimL and Python in
" function definition.
function! Reddit()

" We start the python code like the next line.

python << EOF
# the vim module contains everything we need to interface with vim from
# python. We need urllib2 for the web service consumer.
import vim, urllib2
# we need json for parsing the response
import json

# we define a timeout that we'll use in the API call. We don't want
# users to wait much.
TIMEOUT = 20
URL = "http://reddit.com/.json"

try:
    # Get the posts and parse the json response
    response = urllib2.urlopen(URL, None, TIMEOUT).read()
    json_response = json.loads(response)

    posts = json_response.get("data", "").get("children", "")

    # vim.current.buffer is the current buffer. It's list-like object.
    # each line is an item in the list. We can loop through them delete
    # them, alter them etc.
    # Here we delete all lines in the current buffer
    del vim.current.buffer[:]

    # Here we append some lines above. Aesthetics.
    vim.current.buffer[0] = 80*"-"

    for post in posts:
        # In the next few lines, we get the post details
        post_data = post.get("data", {})
        up = post_data.get("ups", 0)
        down = post_data.get("downs", 0)
        title = post_data.get("title", "NO TITLE").encode("utf-8")
        score = post_data.get("score", 0)
        permalink = post_data.get("permalink").encode("utf-8")
        url = post_data.get("url").encode("utf-8")
        comments = post_data.get("num_comments")

        # And here we append line by line to the buffer.
        # First the upvotes
        vim.current.buffer.append("↑ %s"%up)
        # Then the title and the url
        vim.current.buffer.append("    %s [%s]"%(title, url,))
        # Then the downvotes and number of comments
        vim.current.buffer.append("↓ %s    | comments: %s [%s]"%(down, comments, permalink,))
        # And last we append some "-" for visual appeal.
        vim.current.buffer.append(80*"-")

except Exception, e:
    print e

EOF
" Here the python code is closed. We can continue writing VimL or python again.
endfunction

Save the file, source it in vim (:source vimmit.vim) and:

:call Reddit()

Now, the way we call the function is not so elegant. So we define a command:

command! -nargs=0 Reddit call Reddit()

We define the command :Reddit to call the function. After adding this, open a new bufer and do :Reddit . Home page will be loaded in the buffer. The -nargs argument states how many arguments the command will take.

Function Arguments, Eval and Command

Q: How does one access functional arguments?

function! SomeName(arg1, arg2, arg3)
    " Get the first argument by name in VimL
    let firstarg=a:arg1

    " Get the second argument by position in Viml
    let secondarg=a:1

    " Get the arguments in python

    python << EOF
    import vim

    first_argument = vim.eval("a:arg1") #or vim.eval("a:0")
    second_argument = vim.eval("a:arg2") #or vim.eval("a:1")

You can define a function with arbitrary number of arguments by putting "..." instead of argument names. You can access these arguments only by position, and you can mix them with named arguments (arg1, arg2, ...)

Q: How can I call Vim commands from Python?

vim.command("[vim-command-here]")

Q: How to define global variables and access them in VimL and Python?

Global vars are prefixed with g:. If you want to define one in your script, best thing to do is check if it exists and if doesn't define it and assign some default value to it:

if !exists("g:reddit_apicall_timeout")
    let g:reddit_apicall_timeout=40
endif

You can access it from python using the vim module:

TIMEOUT = vim.eval("g:reddit_apicall_timeout")

If you want to override this setting, you can write:

let g:reddit_apicall_timeout=60

in .vimrc .

Additional Notes

VimL is pretty easy once you try it. Remember that print works and everything you can do with python, you can do in here. Here you can find the documentation for the vim python module. Vimdoc is the possibly the only resource you will need when writing vim plugins.

You can also check this IBM developerWorks article .

Now, try to extend "vimmit.vim" so the user is able to choose a subreddit (as a first functional argument).

W3cValidate.vim

2011-01-23T00:55:00Z

W3cValidate.vim is a plugin that enables buffer/url validation using the W3 Validator API.

Usage

Copy w3cvalidate.vim file inside plugin directory, source it or restart Vim. Use

:W3cValidate to validate the current buffer

:W3cValidate "[url_here]" to validate a hosted page

:W3cValidateDT "[doctype]" to validate the buffer using the [doctype] override. Make sure the doctype is a valid doctype

The validator service url can be changed if you run an instance of it localy by setting g:w3_validator_url in .vimrc

The script has API timeout default to 20 seconds. You can change it by setting g:w3_apicall_timeout in .vimrc

Source

Bitbucket GitHub Vim Scripts

Generate960.py

2011-01-15T12:06:00Z

Command line utility script for generating custom 960gs grids (css). It can generate grids will custom width, column number and gutter width. Outputs to file or stdout.

Usage

python generate960.py

Optional Arguments

-h, --help show help message and exit

-w WIDTH, --width WIDTH

Content width in px. Defaults to 960px.

-c COLUMNS, --columns COLUMNS

Number of columns. Defaults to 12.

-g GUTTER, --gutter GUTTER

Gutter width in px. Defaults to 20px.

-f FILE, --file FILE

Name of the file to be saved. If this is not specified the css will be printed to stdout

To generate 1024 grid with 15px gutters and 10 columns:

python generate960.py -w 1024 -c 10 -g 15

Requirements

Jinja2 and Argparse

Source

Bitbucket GitHub

How to easily enable r600g Gallium3D in Ubuntu and Ubuntu derivatives for radeon

2011-01-05T01:47:00Z

Gallium3D is a software library for 3D graphics device drivers developed by VMware. It's the next big thing in linux graphic drivers world. At the moment, it drives the old (sub r300) ATI graphic cards and it is in a very active development for NVidia and newer ATI(AMD) hardware. You can read more about Gallium here

Last three weeks I became obsessed with the open source radeon driver. There is a lot of buzz going on there, a lot of optimizations and development (much expected in kernel 2.6.38) which is pretty much covered on the forums of Phoronix. I decided to take a spin on new drivers, the development versions of Mesa and a newer kernel 2.6.37 from the Ubuntu mainline builds. I have an R700 based ATI HD4650. The gallium driver for this card is r600g. It shows many improvements and optimizations but it's still in development cycle, unlike the driver for older cards r300g which is pretty much the default driver in Meerkat. Because of it's status, r600g is tricky to install especially if you aren't comfortable with building git versions with compiler switches that enable gallium in mesa. But, thanks to the guys at Xorg-edgers PPA the simple mortals can have gallium on Ubuntu as easy as installing a video player.

Xorg-edgers PPA states that if you install the packages from the main PPA (ppa:xorg-edgers/ppa) and add Option "ForceGallium" "true" in xorg.conf, r600g will be used. However, if this doesn't work with the latest updates, check Xorg-edgers/radeon. Packages from this PPA enable gallium by default. The only thing you need to do is add it:

sudo apt-add-repository ppa:xorg-edgers/radeon
sudo apt-get update && sudo apt-get upgrade

Now if you installed the driver/gl stack from /radeon you should delete xorg.conf if you don't have any monitor specific lines there. If you used the main ppa, you should add that option "ForceGallium" inside the device section and voila, you have r600g driver running.

I tested this with stock kernel 2.6.35 and updated 2.6.37 and it works pretty good. There is huge performance gain which I felt playing Penumbra Overture. On high settings, 1920x1200 resolution, the game is flying. Video is smooth, compiz works great and games play smoother. The 2.6.38 kernel brings even more good news for radeon users. Massive performance gain with the PageFlipping and the CollorTilling implementations in the radeon driver give it a big boost. We just have to wait for a more stable version of the kernel. The kernel is at rc2 stage at the moment of writing.

Anyway, Kudos to the open source driver developers, better make that to ALL open source developers involved in Linux.

Disclaimer: This post is a general guideline for installing the Gallium3D radeon driver for linux practiced by the author. The author will not accept any liability for any damages/data-loss caused using the technique described above.

PyFaceDetect

2010-12-28T18:00:00Z

PyFaceDetect is a thin wrapper around OpenCV for detection and marking faces in images.

The module can be used as an API or in console for testing purposes. For usage and options:

python pyfacedetect.py -h

Sample input

Sample output

The module consists of 2 classes:

OcvDetector that wraps OpenCV Api for face detection and can be used bare without the helper methods for loading images, marking faces, or getting human friendly output

FaceDetect - full feature class that extends OcvDetector and adds helper methods for loading/marking/outputing/saving images and dumping faces in json.

You can enable scaning for profiles for quantity of faces by setting SCAN_FOR_PROFILES to True. That, however will give some overlapping rectangles.

Try and tweak MIN_FACE_SIZE, HAAR_SCALE and MIN_NEIGHBORS so you can change the accuracy of the detection. For more info, read http://opencv.willowgarage.com/documentation/python/objdetect_cascade_classification.html

For speed, all images above 1000width or 1000h are scaled to 1000w or 1000h max.

API Usage

face_detect = new FaceDetect()

face_detect.image_from_file(file_name)#load from file

face_detect.image_from_input(input_id)#load 1 frame from video input

face_detect.detect_faces()#stores faces in local list

face_detect.overlay_image(rgb_border = (255,0,0), width = 2) #overlays original image with rectangles around detected faces.

face_detect.to_json()#returns json string with faces rectangles

face_detect.output_image()#shows image in window

face_detect.save_image(file_name)#saves overlaid image to file_name

Dependencies

OpenCV 2.1+ and Argparse

Source

Bitbucket GitHub

Calculating image entropy with Python, How and Why?

2010-10-04T12:20:00Z

The Why?

Lets say you are developing a video portal, kinda like youtube/vimeo and you want to have an automated way of making thumbnails. Since video manipulation in python is a bit tricky, you might as well make the thumbnails with ffmpeg every [n] seconds. You need 10 thumbnails for each video, and you want those thumbnails to show parts of it.

If you leave it random, you might end up having thumbnails that show a black scene or some scene that barely shows anything. Using the image entropy, you can sort out the thumbnails based on the "business" of the scene they depict. The technique can be implemented as follows:

Generate n>[thumbnails_needed] thumbnails
Calculate the image entropy for each of them
Sort them by their entropy
Use the first [thumbnails_needed] thumbnails

The How?

Entropy H of a sampled signal of length N samples is calculated:

H(X) = -1 * sum_1_to_N ( p_i log( p_i ) )

p_i is the probability of the i-th sample of the signal and can be calculated:

p_i = Histogram(sample)[i]/Length(Histogram(sample))

All we need to do is to get the histogram list and we can calculate the entropy.

On a first sight, this sounds like a complex mathematical problem. And it is, but we will make a little shortcut. We will use the Python imaging library or PIL. This package is de facto standard package for image manipulation in Python. It is one of the first packages I install when preparing my development environment (before making virtualenvs). PIL provides a method for calculating the histogram of the image and solves most of our problems. The function for calculating the entropy looks like this:

import Image
import math


def image_entropy(img):
    """calculate the entropy of an image"""
    histogram = img.histogram()
    histogram_length = sum(histogram)

    samples_probability = [float(h) / histogram_length for h in histogram]

    return -sum([p * math.log(p, 2) for p in samples_probability if p != 0])

img = Image.open('headshot.jpg')
print image_entropy(img)

As you can see, PIL’s method for calculating the histogram really simplifies things and makes the functions seem slim and simple. All its left is iterating through set of thumbnails, calculating the entropy and sorting them.

Remember: Bigger entropy means more noise/liveliness/color/business.* You will usually need the thumbnails with greater entropy.

Install Proprietary ATI 10.8 drivers on Ubuntu Lucid / Linux Mint

2010-09-09T20:55:00Z

I will digress a bit from the theme of the blog, but seeing a lot of people having issues with installing the ATI drivers on Ubuntu, I decided to put my thoughts and experience with it here.

I have experimented with the drivers quite a bit. I own Sapphire Radeon HD 4650, and always try out games on Wine(i wish all of them worked in Gnu/Linux) which is the reason I need updated drivers on my rig. Two weeks ago, AMD(then ATI) updated the drivers to 10.8 version. By their claims, these drivers introduce some performance gain, bugfixes and general stability. I installed the driver 30 minutes before writing this post and it actually works pretty good.

Now for the technical part:

What you need is a rig with AMD graphic card powered by a Linux Mint or Ubuntu Lucid distro with installed build-essential. Driver (which in fact is shell script) can be downloaded Here. The script is called ati-driver-installer-[version]-[CPU architecture].run. Download it to new empty folder - lets call it ati in home folder.

Now we have the driver and we can start with installation. Before we actually commence the installation, we have to purge/remove the old ati driver from the system. You should try these commands to remove it:

sudo apt-get purge fglrx

sudo sh /usr/share/fglrx/fglrx-uninstall.sh

After the removing is done, reboot the PC. You could be asked to turn the graphics to low details (safe mode). Do that. Next, open a terminal and navigate to the folder with the driver.

cd ~/ati/

Add permissions for execution to the script:

chmod +x ati-driver-installer-[version]-[CPU architecture].run

Generate distro-specific packages(for lucid or mint 9):

sudo ./ati-driver-installer-[version]-[CPU architecture].run –buildpkg Ubuntu/lucid

or list all available distro builds:

sudo ./ati-driver-installer-[version]-[CPU architecture].run –listpkg

After this operation (if everything went ok) you will have several .deb packages. Install them with the next command:

sudo dpkg -i *.deb

The drivers are installed. If this is your first ATI driver installation, invoke next command to make the initial config:

sudo aticonfig –initial

Reboot the PC, and you’ll have the drivers installed. If you have any problems, refer to the Installation Instructions on the driver’s download page.

Disclaimer: This post is a general guideline for installing the proprietary ati drivers for linux practiced by the author. The author will not accept any liability for any damages/data-loss caused using the technique described above.

Pgbouncer Makes a Difference

2010-06-15T23:17:00Z

Last week I was building VMware images for database and web server appliances that would host a fairly large Django application. The application is backed by a PostgreSQL database and I was looking for some info on compiling/configuring pgpool on Debian (I like to compile stuff when I can, especially when the last version of Debian is 2 years old). Googling around I came across some very interesting posts on mailing lists and SO regarding Django, PostgeSQL and connection pooling.

Among other things, people seem to have a notion that using pooling middleware won’t accomplish much as the web server still needs to open a TCP connection and that is the source of a noticeable overhead for each request. So they’ve come up with solutions to avoid opening TCP connections as much as possible, trying to accomplish something similar to what SQLAlchemy’s connection pool does, keeping the web server connected to the database with multiple connections at all times. These solutions of course range from changing the code in django.db.backends.... to monkey-patching it.

When you change Django’s code you’ve just created a fork of a growing and evolving open source project and based your own project around that fork you have to maintain yourself. Monkey-patching is not as bad, but comes very close regarding maintenance problems. And is all that really necessary?

The Overhead Doesn’t Stem From Opening TCP Connections

Every time you open a database connection (session) to execute some SQL on a Postgres database the Postgres server spawns a new process and upon closing the connection (session) from your application that process is shut down. With the way Django handles database sessions this is repeated for every request. Which means for every request Postgres will have to spawn a new worker process that will last for the duration of the database session involved in responding to that HTTP request. The overhead involved in opening a TCP connection to a process running on the same machine or on the same network is not much compared to the overhead involved in spawning a new process.

Solutions Do Exist

So if the overhead for each request comes from spawning new processes then the obvious solution would be to keep that at a minimum level. If you keep the connections to the database server open and reuse them for every request then the processes spawned at the time the connections were established would be reused as the connections are reused.

But you don’t have to keep your web server connected to the database server to achieve this. Two more popular solutions are pgpool II and pgbouncer. Both are designed as sort of middleware proxies that sit between your application and your database. Pgpool is more of a replication and load balancing solution than a connection pool. It works as a connection pool because at each connection opened by your application to pgpool it will have a separate process handling that connection, but it will keep those processes alive and connected to the Postgres server even after your application closes those connections. So using it would have the effect of lowering the net amount of new processes created to serve a certain number of requests to your web application. Pgbouncer on the other hand handles all the requests between your application and Postgres in a highly efficient asynchronous manner by utilizing libevent and not using multiprocessing at all, and it will keep the initially opened connections for a longer time after your application closes them, so making a new connection to pgbouncer will rarely result in Postges spawning a process.

The Proof

To prove that solutions like pgbouncer do make a difference I created a simple test scenario. A very small Django project with one page displaying 5 rows from a table in a PostgreSQL database. Both the database server and the web server (Apache with mod-wsgi in daemon mode) running on one small VM with 1GB of RAM and 4 CPU cores assigned.

In the first test I configured the application to connect to the database server directly and put the page under siege with 1, 5, 15,

50, 100 and 200 concurrent requests, each session lasting for 1 minute. Then I repeated the process with the application configured to connect to pgbouncer instead. The results show something close to a 50% increase in responsiveness:

Django Toolbox for Komodo

2010-05-03T16:08:00Z

If you’re using TextMate for Django development you’ve probably came across the Python Django and Python Django Template bundles.

The Django Toolbox for Komodo does most of what the TextMate bundles do, plus integrated shortcuts for the most important links in the Django docs. Builtin template filter support is still a work in progress.

To get the most out of the toolbox, first install the Tab Abbreviations add-on and set a keyboard shortcut (eg. TAB) to the command “Insert Abbreviation Snippet By Name”.

Download link: DjangoKomodoToolbox-0.1.kpz

JqCarousel Three-D

2010-01-27T19:59:00Z

This plugin was coded to prove a point. We saw another similar plugin like this, but it was only for sale with obfuscated code, so I decided to write it and opensource it. It doesn't have the limited functionality and the custom implementation of the paid ones, it is easy to implement and highly customizable. Works with images and other HTML containers.

Here you can check out a demo of the carousel. The JS embed code is:

$('#carouselthreed').carousel3D({
    speed: 900,
    perspectiveZoom: 70,
    sideOffset: 80,
    topOffset:0,
    secondaryOpacity: .35,
    emClass:'tdc-element'
})

And here is the HTML:

<div id="carouselthreed">
    <img class="tdc-element" src="fly.jpg">
    <img class="tdc-element" src="gates.jpg">
    <img class="tdc-element" src="penguins.jpg">
</div>

The settings object attributes are: - speed - Speed of animation in milliseconds (default 350) - perspectiveZoom - Percentage of secondary element's sizes relative to primary element (default 80) - sideOffset - Offset of secondary elements (default 70) - topOffset - Offset from top of secondary elements (default 0) - secondaryOpacity - Opacity of secondary elements (default 0.8) - emClass - CSS class of elements that will rotate (images/dom) (default "tdc-element")

Any of those attributes can be omitted and default values will be used. For reference, the second element in the container is taken, it is the central element and all other elements are resized relative to it.

It works with unlimited number of elements.

Download the plugin and get a fresh copy of JQuery and you're set.

Enjoy

Benchmark of django deployment techniques

2010-01-19T19:55:00Z

I made a benchmark of different Django deployment techniques and configurations mostly for my personal purposes. The results are published in the hope that it would save others some time. The benchmark was not designed to test the speed of Django itself, only to give relative comparison between different techniques for running a Django application in production environments. I'm not affiliated with any of the following open source projects: Apache, Nginx, Cherokee, mod_wsgi, mod_python, Cherrypy or uWSGI. Further, I'm not claiming to be an expert in configuring any of the software mentioned here.

What was measured

The Django project used for the benchmark was a simple application for displaying rows from 3 different tables with pagination. Each page had references to 3 static files (css, javascript and an image). Each page involved rendering a simple template inheriting from a base template and including another one, built-in filters were also used. The database had more than a million records in all three tables combined. I browsed different pages of the application over a proxy which recorded the URLs of the browsing session. So for each request that was handled by the application there were 3 more requests for static files. For each deployment technique I ran 4 tests at different concurrency levels for 1 minute making GET requests to the recorded URLs. For each test run I recorded throughput (number of requests served per second), response time (average time in which a request was served) and longest request (the longest time a request was served in each run). Only for the tests at highest concurrency levels I recorded memory usage. I tried to make sure that only the necessary processes for each test were running at a time. Automatic maintenance tasks on the system and the database were turned off. Every test cycle was repeated at least 3 times to recheck the results.

Hardware and software details

For generating the loads I used Siege and ran all the benchmarks over gigabit ethernet from a 2.16 GHz machine with 2 GB of RAM running OS X Snow Leopard.

The system that served as a web and database server ran in a VMware appliance on a 2.8 GHz Core2Duo PC with 8GBs of RAM. The appliance was given only 1GB of working memory and assigned both cores of the CPU. Software details:

Ubuntu 9.10 32b
Python 2.6
PostgreSQL 8.4
Apache 2.2.12 worker MPM
Nginx 0.7.64
Cherokee 0.99.39
Django 1.1
psycopg2 2.0.8

Tested configurations

Apache with mod_wsgi

This was the first configuration I tested. Apache was serving both the static files and dynamic content via mod_wsgi which ran in daemon mode with 5 processes and 1 thread per process.

Nginx + Apache with mod_wsgi

My preferred configuration for running django sites. Apache with mod_wsgi was used only for the dynamic content, requests to these urls were proxied by nginx. Static files were served by nginx directly.

Nginx + fcgi

Here nginx is used for serving the static content while the dynamic content was handled by FastCGI processes. I used all the defaults from the runfcgi management command and used a TCP socket instead of a socket file because I was bumping into issues with access to the socket file at large numbers of concurrent requests.

Cherokee + SCGI

This was set up from the Cherokee web based wizard for deploying django applications. Static files were served by Cherokee directly. I have to say this is by far the easiest method of deploying Django applications in production environments.

Cherokee + Apache with mod_wsgi

This is essentially the same as Nginx + Apache and mod_wsgi except here Cherokee was used as a proxy and for serving static content. Since it's the first time I'm using Cherokee everything was configured via the web based admin interface and all parameters were left to default values.

Nginx + Cherrypy WSGI server

Here I used the django-cpserver management command to run the application in the Cherrypy WSGI server. 5 Instances of the WSGI server were running behind Nginx as a load balancer. Nginx was serving the static files.

Configurations that are left out

Apache with mod_python

This was left out because I couldn't get consistent results at 250 concurrent requests and the application would often error out at this concurrency level. The benchmark already took a significant amount of my time and I'm not experienced with mod_python so I decided not to proceed with locating the problem or publishing any shaky results.

uWSGI

I tried to deploy the application on uWSGI with the Cherokee web based wizard. With the default configuration (1 process) the tests ran 4 times slower at high concurrency compared to the other configurations. Bumping up the number of process to 5 still didn't yield comparable results. At 25 processes I got comparable results but memory usage skyrocketed. I still suspect I was doing something wrong here, so I didn't publish the results.

lighttpd

Personally I avoid using lighty for a number of subjective reasons. When I have the time I may update this post with some configurations based on lighty, although I wouldn't expect the results to be much different than the ones for the Nginx or Cherokee configurations.

The results

Memory usage

I measured memory usage only at 250 concurrent requests. All the tests ran for 60 seconds, so you can work out where the load on the server happened from the graphs.

Apache with mod_wsgi

Nginx + Apache with mod_wsgi

Nginx + FCGI

Nginx + Cherrypy

Cherokee + SCGI

Cherokee + Apache with mod_wsgi

Announcing Django-audit-log

2009-12-22T21:39:00Z

For those unfamiliar with the term, a definition from Wikipedia:

Audit trail or audit log is a chronological sequence of audit records, each of which contains evidence directly pertaining to and resulting from the execution of a business process or system function.

How does this come into play in a web application?

Lets examine the case of a simple application for keeping track of a store's inventory:

In the most simple case there would be a single database table in which we'd keep details on different products in the store. In the case where multiple users would have access to INSERT/UPDATE/DELETE records in the products table, one user could insert a product with name, description and price, later another user could change the description or even delete the whole record. If at some later point we wanted to restore the original record or see who made the latest changes we'd have to ask all the users to remember what they did. An audit log for this table would provide the means of keeping track of all the changes that were made to it and who made the changes in a chronological order.

django-audit-log provides such facilities for your Django models. It's designed to be very simple to add chronological tracking to any django model with the least amount of changes to your existing code. Adding an audit log for your models is done in three steps:

Add a middleware class in settings.py.
Add a manager property to every model you need to keep track of.
Execute the syncdb management command.

To keep track of all the changes a separate table will be created for each tracked model. This table would have the same column structure as the original model plus columns for tracking the time, type of action (create, change or delete) and user who did the action. Queries on the audit log for a model are made via the manager added in step 2.

The project is still under heavy development and there's no official release yet. Keep that in mind if you consider using it in production. The code can be downloaded from the mercurial repository:

hg clone https://django-audit-log.googlecode.com/hg/ django-audit-log

Basic usage instructions can be found on this wiki page . Feature requests are always welcome.

Details on how it works and extension points will be coming up on the project wiki page soon.

Livescape jQuery Live Landscape

2009-12-15T19:56:00Z

The idea behind this plugin is to have some kind of landscape container that will contain objects moving inside of it e.g. sky with the clouds moving. The plugin is built simple, has simple implementation, it is very easy to use, yet very flexible. You only need the landscape image, the object images and one container.

$('#livescape').livescape({
    //height of the landscape (can be omitted)
    height : '360px',
    //width of the landscape (can be omitted)
    width : '1000px',
    //background image (can be omitted)
    background_image : 'img/bg.jpg',

    // Next define the objects that are moving:
    objects : [
        {
            //the positions are relative to the landscape
            //if omitted, random assumed - same for duration

            //x of start position
            start_x : "1700px",
            //y of start position
            start_y : "190px",
            //x of end position
            end_x : "-4000px",
             //y of end position
            end_y : "190px",
            //image of the object (url)
            image : 'img/car.png',
            // duration of the animation
            duration : 18000,
            // the animation loops
            loop : true,
            //the object is anchor
            href : "ThisisBatCountry.png",
            //pause in between loops in ms
            pause: 1000,
            //the objects fade in on start and out on end
            fade : true
        },
        //another object
        {
            start_x : "-250px",
            start_y : "5px",
            end_x : "1250px",
            end_y : "5px",
            duration : 49000,
            image : 'img/cloud1.png',
            loop : true,
            fade : true
        }
    ]
})

This is setup is for 2 objects. The implementation is pretty intuitive and i think you will get a hold of it very fast.

Update

The objects now turn towards the direction of movement using CSS3 rotation.
For every object you can define a random starting and ending range using the attrubutes:
- range_sx_min (start_x min range)
- range_sx_max (start_x max range)
- range_ex_min
- range_ex_max
- range_sy_min
- range_sy_max
- range_ey_min
- range_ey_max
all these attributes can be omitted and random will be used.

The plugin is still beta and needs refactoring. Nevertheless you will have access to an early access version.

Source and example

You can check this plugin in action here

Get the source

`-h, --help`	show help message and exit
`-w WIDTH, --width WIDTH`
	Content width in px. Defaults to 960px.
`-c COLUMNS, --columns COLUMNS`
	Number of columns. Defaults to 12.
`-g GUTTER, --gutter GUTTER`
	Gutter width in px. Defaults to 20px.
`-f FILE, --file FILE`
	Name of the file to be saved. If this is not specified the css will be printed to stdout