Skip to content

Redirecting the stdout on a gtk.TextView

27/07/2009

The problem

Suppose you want to spawn a system process in python and you want to intercept the stdout of the process and redirecting it on a gtk.TextView.
For example in this tutorial you will see how to redirect a generic command ( in the example there is a typical blocking process).

The solution

The most correct solution of the problem is not so immediate, if you search around you will get some answers related to threads and so on… If you study threads is a good thing but we can do it without knowing threads :). Expecially because threads in python can be buggy in some particular cases ( like in pygtk itself).

Spawning a process

To spawn a process the best solution ever is to use the subprocess module. The syntax is simple:

  • popen_object = subprocess.Popen(command, shell = True, stdout = subprocess.PIPE)
  • We tell to subprocess to spawn a process with the command “command”
  • The option shell = True tell subprocess that the command is a shell command
  • We also want to redirect the stdout to a pipe  ->  that is the popen_object.
import subprocess
proc = subprocess.Popen("ls ../", shell = True,stdout=subrocess.PIPE) # ls ../ is an example

The proc object is a Popen object, it has some interesting methods and attributes, in particular we can access
the stdout of the process,  (like) it’s a file.

Interesting methods and attributes:

  • proc.communicate() will wait until the process finish and give us the stdout
  • proc.kill() will kill the process
  • proc.stdout is a file descriptor (an opened file) that we can use

There are a lot of other features, if you are interested (maybe you are) look at the documentation.

Redirecting the stdout

To read and redirect the stdout of the process to the TextView widget we will use the glib library that’s shipped with pygtk. This library has a function glib.io_add_watch that is pretty smart:

glib.io_add_watch

    def glib.io_add_watch(fd, condition, callback, ...)
fd : a Python file object or an integer file descriptor ID
condition : a condition mask
callback : a function to call
... : additional arguments to pass to callback
Returns : an integer ID of the event source

The glib.io_add_watch() function arranges for the file (specified by fd) to be monitored by the main loop for the specified conditionfd may be a Python file object or an integer file descriptor. The value of condition is a combination of:

glib.IO_IN There is data to read.
glib.IO_OUT Data can be written (without blocking).
glib.IO_PRI There is urgent data to read.
glib.IO_ERR Error condition.
glib.IO_HUP Hung up (the connection has been broken, usually for pipes and sockets).

Additional arguments to pass to callback can be specified after callback. The idle priority may be specified as a keyword-value pair with the keyword “priority”. The signature of the callback function is:

  def callback(source, cb_condition, ...)

where source is fd, the file descriptor; cb_condition is the condition that triggered the signal; and, ... are the zero or more arguments that were passed to the glib.io_add_watch() function.

If the callback function returns FALSE it will be automatically removed from the list of event sources and will not be called again. If it returns TRUE it will be called again when the condition is matched.

  • The fd is an opened file like our  proc.stdout!!
  • the condition is something like “there’s something to read”:  in computer language glib.IO_IN
  • the callback is a normal function with this prototype:  callback(fd, condition_triggered), the condition triggered tell us why the callback was called (there’s something to read, someone has interrupted the process and so on…)
  • NOTE: this particular callback has to return True or False, from the reference:
    “If the callback function returns FALSE it will be automatically removed from the list of event sources and will not be called again. If it returns TRUE it will be called again when the condition is matched.”

Creating you smart TextView

the plan is to create a special textview, we pass a command, and this Textview presents itself giving us the command stdout. Well we know that her constructor has one argument:

#Test driven developement, write the test:
def test():
    # This command will download one project of mine deheh
    ctv = CommandTextView("svn co https://fitta.svn.sourceforge.net/svnroot/fitta fitta")
    win=gtk.Window()
    win.connect("delete-event", lambda wid,event: gtk.main_quit()) # Defining callbacks with lambdas
    win.set_size_request(200,300)
    win.add(ctv)
    win.show_all()
    ctv.run()
    gtk.main()
if __name__=='__main__' : test()

We need also a method run that starts all the things, now let’s write the class, in the constructor simply
we will save the command as a variable and we’ll do the initialization stuff,  in the run we will use subprocess to spawn the process and we will use glib.io_add_watch to fetch the stdout.

import gtk,glib
import subprocess

class CommandTextView(gtk.TextView):
    ''' Nice TextView that reads the output of a command syncronously '''
    def __init__(self, command):
        '''command : the shell command to spawn'''
        super(CommandTextView, self).__init__()
        self.command = command
    def run(self):
        ''' Runs the process '''
        proc = subprocess.Popen(self.command, stdout = subprocess.PIPE) # Spawning
        glib.io_add_watch(proc.stdout, # file descriptor
                          glib.IO_IN,  # condition
                          self.write_to_buffer ) # callback
    def write_to_buffer(self, fd, condition):
        if condition == glib.IO_IN: #if there's something interesting to read
           char = fd.read(1) # we read one byte per time, to avoid blocking
           buf = self.get_buffer()
           buf.insert_at_cursor(char) # When running don't touch the TextView!!
           return True # FUNDAMENTAL, otherwise the callback isn't recalled
        else:
           return False # Raised an error: exit and I don't want to see you anymore

Every time there is something to read the glib.io_add_watch recall the callback,
and in the callback we write to the textview, nice and simple.

For the complete example, you can download at this link:

command-textview.py

About these ads
7 Comments leave one →
  1. S Smith permalink
    01/05/2012 5:34 pm

    I really appreciated this post. However, I came across the problem that the output of the generic command was buffered when sent to a pipe. My solution was to use pty:


    import pty
    import subprocess,
    import os
    master, slave = pty.openpty()
    proc = subprocess.Popen([cmd],stdout=slave,stderr=slave)
    gobject.io_add_watch(os.fdopen(master),gobject.IO_IN,write_to_buffer)

    In write_to_buffer, I had to use fd.readline, instead of fd.read(1) to get all of the output.

    Also, if you put the TextView in a ScrolledWindow, the following code keeps the newest output (at the bottom of the TextView) in view.


    sw = gtk.ScrolledWindow()
    vadj = sw.get_vadjustment()
    vadj.connect('changed',rescroll,sw)
    def rescroll(adj,scroll):
    """
    Scroll to the bottom of the TextView when the adjustment changes.
    """
    adj.set_value(adj.upper-adj.page_size)
    scroll.set_vadjustment(adj)

    -Sterling

  2. Marc Rechté permalink
    25/03/2010 1:37 pm

    Hello,

    On http://library.gnome.org/devel/pygobject/stable/ io_add_watch is in glib module…

    When running a similar above example I get Warning: g_io_channel_unix_new: 3 is neither a file descriptor or a socket (when calling the glib.io_ass_watch(proc.stdout, …) function).

    Is that because I call this function in another thread (not the one running the gtk main loop) ?

    Thanks

    • 25/03/2010 2:18 pm

      Hi! I’m actually reading the link you sent me,
      ”Many functions that previously were in this namespace got moved to glib namespace instead. They are still available in gobject for backward compatibility, but not documented here.” In the initial version of this tutorial I used the glib module, then I changed because of the comment you can see below, well, it’s time for changing it another time! (Thank you for point this out…).

      About your problem I can’t tell you much. IMHO it’s not related to threads, here’s the documentation, it seems related to the fact that something bad was happened to your file descriptor (The glib.io_add_watch does a lot of checks, like if the file is closed etc…). But I can’t help too much. However I can investigate further if you send me your example code.

      • Marc Rechté permalink
        25/03/2010 3:05 pm

        Thanks for your prompt answer and help proposal. I forgot to tell you that the host OS is Windows (Gtk+ 2.16.2). I tested on Linux Fedora (Gtk+2.14) and it works fine ! On the document link you sent, there is a mention regarding Windows, but it may not be applicable here.

  3. 26/11/2009 2:39 am

    It seems glib.io_add_watch and glib.IO_IN is change to gobject.io_add_watch and gobject.IO_IN in PyGTK 2.0 and Python 2.6

  4. Yves permalink
    22/11/2009 9:21 pm

    Hello Gabriele

    Grazie per il tutorial
    I had this problem of redirection and I googled a lot to find a solution (you had not yet posted this tutorial !)
    I used suprocess.Popen and gobject.idle_add.
    Your solution with glib.io_add_watch is more elegant (looks like the linux/unix select() system call more familiar to me)
    I will adjust my code using this.

    Nice blog.
    Ivo, from Toulouse, Francia.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Follow

Get every new post delivered to your Inbox.

%d bloggers like this: