Python utility like the Unix cut command - Part 1 - cut1.py

By Vasudev Ram

Regular readers of my blog might have noticed that I sometimes write (and write about) command line utilities in posts here.

Recently, I thought of implementing a utility like the Unix cut command in Python.

Here is my first cut at it (pun intended :), below.

However, instead of just posting the final code and some text describing it, this time, I had the idea of doing something different in a post (or a series of posts).

I thought it might be interesting to show some of the stages of development of the utility, such as incremental versions of it, with features or code improvements added in each successive version, and a bit of discussion on the design and implementation, and also on the thought processes occurring during the work.

(For beginners: incidentally, speaking of thought processes, during interviews for programming jobs at many companies, open-ended questions are often asked, where you have to talk about your thoughts as you work your way through to a solution to a problem posed. I know this because I've been on both sides of that many times. This interviewing technique helps the interviewers gauge your thought processes, and thereby, helps them decide whether they think you are good at design and problem-solving or not, which helps decide whether you get the job or not.)

One reason for doing this format of post, is just because it can be fun (for me to write about, and hopefully for others to read - I know I myself like to read such posts), and another is because one of my lines of business is training (on Python and other areas), and I've found that beginners sometimes have trouble going from a problem or exercise spec to a working implementation, even if they understand well the syntax of the language features needed to implement a solution. This can happen because 1) there can be many possible solutions to a given programming problem, and 2) the way to break down a problem into smaller pieces (stepwise refinement), that are more amenable to translating into programming statements and constructs, is not always self-evident to beginners. It is a skill one acquires over time, as one keeps on programming for months and years.

So I'm going to do it that way (with more explanation and multiple versions), over the course of a few posts.

For this first post, I'll just describe the rudimentary first version that I implemented, and show the code and the output of a few runs of the program.

The Wikipedia link about Unix cut near the top of this post describes its behavior and command line options.

In this first version, I only implement a subset of those:
- reading from a file (only a single file, and not reading from standard input (stdin))
- only support the -c (for cut by column) option (not the -b (by byte) or -f (by field) options)
- only support one column specification, i.e. -cm-n, not forms like -cm1-n1,m2-n2,...

In subsequent versions, I'll add support for some of the omitted features, and also fix any errors that I find in previous versions, by testing.

I'll call the versions cutN.py, where 1 <= N <= the highest version I implement. So this current post is about cut1.py.

Here is the code for cut1.py:

"""
File: cut1.py
Purpose: A Python tool somewhat similar to the Unix cut command.
Does not try to be exactly the same or implement all the features 
of Unix cut. Created for educational purposes.
Author: Vasudev Ram
Copyright 2017 Vasudev Ram
Web site: https://vasudevram.github.io
Blog: https://jugad2.blogspot.com
Product store: https://gumroad.com/vasudevram
"""

from __future__ import print_function

import sys
from error_exit import error_exit

def usage(args):
    #"Cuts the specified columns from each line of the input.\n", 
    lines = [
    "Print only the specified columns from lines of the input.\n", 
    "Usage: {} -cm-n file\n".format(args[0]), 
    "or: cmd_generating_text | {} -cm-n\n".format(args[0]), 
    "where -c means cut by column and\n", 
    "m-n means select (character) columns m to n\n", 
    "For each line, the selected columns will be\n", 
    "written to standard output. Columns start at 1.\n", 
    ]
    for line in lines:
        sys.stderr.write(line)

def cut(in_fil, start_col, end_col):
    for lin in in_fil:
        print(lin[start_col:end_col])

def main():

    sa, lsa = sys.argv, len(sys.argv)
    # Support only one input file for now.
    # Later extend to support stdin or more than one file.
    if lsa != 3:
        usage(sa)
        sys.exit(1)

    prog_name = sa[0]

    # If first two chars of first arg after script name are not "-c",
    # exit with error.
    if sa[1][:2] != "-c":
        usage(sa)
        error_exit("{}: Expected -c option".format(prog_name))

    # Get the part of first arg after the "-c".
    c_opt_arg = sa[1][2:]
    # Split that on "-".
    c_opt_flds = c_opt_arg.split("-")
    if len(c_opt_flds) != 2:
        error_exit("{}: Expected two field numbers after -c option, like m-n".format(prog_name))

    try:
        start_col = int(c_opt_flds[0])
        end_col = int(c_opt_flds[1])
    except ValueError as ve:
        error_exit("Conversion of either start_col or end_col to int failed".format(
        prog_name))

    if start_col < 1:
        error_exit("Error: start_col ({}) < 1".format(start_col))
    if end_col < 1:
        error_exit("Error: end_col ({}) < 1".format(end_col))
    if end_col < start_col:
        error_exit("Error: end_col < start_col")
    
    try:
        in_fil = open(sa[2], "r")
        cut(in_fil, start_col - 1, end_col)
        in_fil.close()
    except IOError as ioe:
        error_exit("Caught IOError: {}".format(repr(ioe)))

if __name__ == '__main__':
    main()

Here are the outputs of a few runs of cut1.py. I used this text file for the tests.
The line of digits at the top acts like a ruler :) which helps you know what character is at what column:

$ type cut-test-file-01.txt
12345678901234567890123456789012345678901234567890
this is a line with many words in it. how is it.
here is another line which also has many words.
now there is a third line that has some words.
can you believe it, a fourth line exists here.

$ python cut1.py
Print only the specified columns from lines of the input.
Usage: cut1.py -cm-n file
or: cmd_generating_text | cut1.py -cm-n
where -c means cut by column and
m-n means select (character) columns m to n
For each line, the selected columns will be
written to standard output. Columns start at 1.

$ python cut1.py -c
Print only the specified columns from lines of the input.
Usage: cut1.py -cm-n file
or: cmd_generating_text | cut1.py -cm-n
where -c means cut by column and
m-n means select (character) columns m to n
For each line, the selected columns will be
written to standard output. Columns start at 1.

$ python cut1.py -c a
cut1.py: Expected two field numbers after -c option, like m-n

$ python cut1.py -c0-0
Print only the specified columns from lines of the input.
Usage: cut1.py -cm-n file
or: cmd_generating_text | cut1.py -cm-n
where -c means cut by column and
m-n means select (character) columns m to n
For each line, the selected columns will be
written to standard output. Columns start at 1.

$ python cut1.py -c0-0 a
Error: start_col (0) < 1

$ python cut1.py -c1-0 a
Error: end_col (0) < 1

$ python cut1.py -c1-1 a
Caught IOError: IOError(2, 'No such file or directory')

$ python cut1.py -c1-1 cut-test-file-01.txt
1
t
h
n
c

$ python cut1.py -c6-12 cut-test-file-01.txt
6789012
is a li
is anot
here is
ou beli

$ python cut1.py -20-12 cut-test-file-01.txt
Print only the specified columns from lines of the input.
Usage: cut1.py -cm-n file
or: cmd_generating_text | cut1.py -cm-n
where -c means cut by column and
m-n means select (character) columns m to n
For each line, the selected columns will be
written to standard output. Columns start at 1.
cut1.py: Expected -c option

$ python cut1.py -c20-12 cut-test-file-01.txt
Error: end_col < start_col

- Vasudev Ram - Online Python training and consulting

Get updates (via Gumroad) on my forthcoming apps and content.

Jump to posts: Python * DLang * xtopdf

Subscribe to my blog by email

My ActiveState Code recipes

Follow me on: LinkedIn * Twitter

Are you a blogger with some traffic? Get Convertkit:

Email marketing for professional bloggers

Share |

Quick CMD one-liner like "which python" command - and more

By Vasudev Ram

Today, while browsing StackOverflow, I saw this question:

Is there an equivalent of 'which' on the Windows command line?

And I liked the second answer.

The first part of that answer had this solution:

c:\> for %i in (cmd.exe) do @echo.   %~$PATH:i
   C:\WINDOWS\system32\cmd.exe

c:\> for %i in (python.exe) do @echo.   %~$PATH:i
   C:\Python25\python.exe

Then I modified the one-liner above to add a few more commands to search for, and ran it:

$ for %i in (cmd.exe python.exe metapad.exe file_sizes.exe tp.exe alarm_clock.py
) do @echo.   %~$PATH:i
   C:\Windows\System32\cmd.exe
   D:\Anaconda2-4.2.0-32bit\python.exe
   C:\util\metapad.exe
   C:\util\file_sizes.exe
   C:\util\tp.exe
   C:\util\alarm_clock.py

Notice that I also included a .py file in the list of commands to search for, and it worked for that too. This must be because .py files are registered as executable files (i.e. executable by the Python interpreter) at the time of installing Python on Windows.

Of course, as a user comments in that SO post, this is not exactly the same as the which command, since you have to specify the .exe extension (for the commands you search for), and there are other extensions for executable files, such as .bat and others. But since the Python interpeter is normally named python.exe, this can be used as a quick-and-dirty way to find out which Python executable is going to be run when you type "python", if you have more than one of them installed on your system.

I had also written a simple Python tool roughly like the Unix which command, earlier:
A simple UNIX-like "which" command in Python

---

- Vasudev Ram - Online Python training and consulting

Get updates (via Gumroad) on my forthcoming apps and content.

Jump to posts: Python * DLang * xtopdf

Subscribe to my blog by email

My ActiveState Code recipes

Follow me on: LinkedIn * Twitter

Are you a blogger with some traffic? Get Convertkit:

Email marketing for professional bloggers

Share |

Show error numbers and codes from the os.errno module

By Vasudev Ram

While browsing the Python standard library docs, in particular the module os.errno, I got the idea of writing this small utility to display os.errno error codes and error names, which are stored in the dict os.errno.errorcode:

Here is the program, os_errno_info.py:

from __future__ import print_function
'''
os_errno_info.py
To show the error codes and 
names from the os.errno module.
Author: Vasudev Ram
Copyright 2017 Vasudev Ram
Web site: https://vasudevram.github.io
Blog: https://jugad2.blogspot.com
Product store: https://gumroad.com/vasudevram
'''

import sys
import os

def main():
    
    print("Showing error codes and names\nfrom the os.errno module:")
    print("Python sys.version:", sys.version[:6])
    print("Number of error codes:", len(os.errno.errorcode))
    print("{0:>4}{1:>8}   {2:<20}    {3:<}".format(\
        "Idx", "Code", "Name", "Message"))
    for idx, key in enumerate(sorted(os.errno.errorcode)):
        print("{0:>4}{1:>8}   {2:<20}    {3:<}".format(\
            idx, key, os.errno.errorcode[key], os.strerror(key)))

if __name__ == '__main__':
    main()

And here is the output on running it:

$ py -2 os_errno_info.py >out2 && gvim out2
Showing error codes and names
from the os.errno module:
Python sys.version: 2.7.12
Number of error codes: 86
 Idx    Code   Name                    Message
   0       1   EPERM                   Operation not permitted
   1       2   ENOENT                  No such file or directory
   2       3   ESRCH                   No such process
   3       4   EINTR                   Interrupted function call
   4       5   EIO                     Input/output error
   5       6   ENXIO                   No such device or address
   6       7   E2BIG                   Arg list too long
   7       8   ENOEXEC                 Exec format error
   8       9   EBADF                   Bad file descriptor
   9      10   ECHILD                  No child processes
  10      11   EAGAIN                  Resource temporarily unavailable
  11      12   ENOMEM                  Not enough space
  12      13   EACCES                  Permission denied
  13      14   EFAULT                  Bad address
  14      16   EBUSY                   Resource device
  15      17   EEXIST                  File exists
  16      18   EXDEV                   Improper link
  17      19   ENODEV                  No such device
  18      20   ENOTDIR                 Not a directory
  19      21   EISDIR                  Is a directory
  20      22   EINVAL                  Invalid argument
  21      23   ENFILE                  Too many open files in system
  22      24   EMFILE                  Too many open files
  23      25   ENOTTY                  Inappropriate I/O control operation
  24      27   EFBIG                   File too large
  25      28   ENOSPC                  No space left on device
  26      29   ESPIPE                  Invalid seek
  27      30   EROFS                   Read-only file system
  28      31   EMLINK                  Too many links
  29      32   EPIPE                   Broken pipe
  30      33   EDOM                    Domain error
  31      34   ERANGE                  Result too large
  32      36   EDEADLOCK               Resource deadlock avoided
  33      38   ENAMETOOLONG            Filename too long
  34      39   ENOLCK                  No locks available
  35      40   ENOSYS                  Function not implemented
  36      41   ENOTEMPTY               Directory not empty
  37      42   EILSEQ                  Illegal byte sequence
  38   10000   WSABASEERR              Unknown error
  39   10004   WSAEINTR                Unknown error
  40   10009   WSAEBADF                Unknown error
  41   10013   WSAEACCES               Unknown error
  42   10014   WSAEFAULT               Unknown error
  43   10022   WSAEINVAL               Unknown error
  44   10024   WSAEMFILE               Unknown error
  45   10035   WSAEWOULDBLOCK          Unknown error
  46   10036   WSAEINPROGRESS          Unknown error
  47   10037   WSAEALREADY             Unknown error
  48   10038   WSAENOTSOCK             Unknown error
  49   10039   WSAEDESTADDRREQ         Unknown error
  50   10040   WSAEMSGSIZE             Unknown error
  51   10041   WSAEPROTOTYPE           Unknown error
  52   10042   WSAENOPROTOOPT          Unknown error
  53   10043   WSAEPROTONOSUPPORT      Unknown error
  54   10044   WSAESOCKTNOSUPPORT      Unknown error
  55   10045   WSAEOPNOTSUPP           Unknown error
  56   10046   WSAEPFNOSUPPORT         Unknown error
  57   10047   WSAEAFNOSUPPORT         Unknown error
  58   10048   WSAEADDRINUSE           Unknown error
  59   10049   WSAEADDRNOTAVAIL        Unknown error
  60   10050   WSAENETDOWN             Unknown error
  61   10051   WSAENETUNREACH          Unknown error
  62   10052   WSAENETRESET            Unknown error
  63   10053   WSAECONNABORTED         Unknown error
  64   10054   WSAECONNRESET           Unknown error
  65   10055   WSAENOBUFS              Unknown error
  66   10056   WSAEISCONN              Unknown error
  67   10057   WSAENOTCONN             Unknown error
  68   10058   WSAESHUTDOWN            Unknown error
  69   10059   WSAETOOMANYREFS         Unknown error
  70   10060   WSAETIMEDOUT            Unknown error
  71   10061   WSAECONNREFUSED         Unknown error
  72   10062   WSAELOOP                Unknown error
  73   10063   WSAENAMETOOLONG         Unknown error
  74   10064   WSAEHOSTDOWN            Unknown error
  75   10065   WSAEHOSTUNREACH         Unknown error
  76   10066   WSAENOTEMPTY            Unknown error
  77   10067   WSAEPROCLIM             Unknown error
  78   10068   WSAEUSERS               Unknown error
  79   10069   WSAEDQUOT               Unknown error
  80   10070   WSAESTALE               Unknown error
  81   10071   WSAEREMOTE              Unknown error
  82   10091   WSASYSNOTREADY          Unknown error
  83   10092   WSAVERNOTSUPPORTED      Unknown error
  84   10093   WSANOTINITIALISED       Unknown error
  85   10101   WSAEDISCON              Unknown error

In the above Python command line, you can of course skip the "&& gvim out2" part. It is just there to automatically open the output file in gVim (text editor) after the utility runs.

The above output was from running it with Python 2.
The utility is written to also work with Python 3.
To change the command line to use Python 3, just change 2 to 3 everywhere in the above Python command :)
(You need to install or already have py, the Python Launcher for Windows, for the py command to work. If you don't have it, or are not on Windows, use python instead of py -2 or py -3 in the above python command line - after having set your OS PATH to point to Python 2 or Python 3 as wanted.)

The only differences in the output are the version message (2.x vs 3.x), and the number of error codes - 86 in Python 2 vs. 101 in Python 3.
Unix people will recognize many of the messages (EACCES, ENOENT, EBADF, etc.) as being familiar ones that you get while programming on Unix.
The error names starting with W are probably Windows-specific errors. Not sure how to get the messages for those, need to look it up. (It currently shows "Unknown error" for them.)

This above Python utility was inspired by an earlier auxiliary utility I wrote, called showsyserr.c, as part of my IBM developerWorks article, Developing a Linux command-line utility (not the main utility described in the article). Following (recursively) the link in the previous sentence will lead you to the code for both the auxiliary and the main utility, as well as the PDF version of the article.

Enjoy.

- Vasudev Ram - Online Python training and consulting

Get updates (via Gumroad) on my forthcoming apps and content.

Jump to posts: Python * DLang * xtopdf

Subscribe to my blog by email

My ActiveState Code recipes

Follow me on: LinkedIn * Twitter

Managed WordPress Hosting by FlyWheel

Share |

An Unix seq-like utility in Python

By Vasudev Ram


Due to a chain (or sequence - pun intended :) of thoughts, I got the idea of writing a simple version of the Unix seq utility (command-line) in Python. (Some Unix versions have a similar command called jot.)

Note: I wrote this program just for fun. As the seq Wikipedia page says, modern versions of bash can do the work of seq. But this program may still be useful on Windows - not sure if the CMD shell has seq-like functionality or not. PowerShell probably has it, is my guess.)

The seq command lets you specify one or two or three numbers as command-line arguments (some of which are optional): the start, stop and step values, and it outputs all numbers in that range and with that step between them (default step is 1). I have not tried to exactly emulate seq, instead I've written my own version. One difference is that mine does not support the step argument (so it can only be 1), at least in this version. That can be added later. Another is that I print the numbers with spaces in between them, not newlines. Another is that I don't support floating-point numbers in this version (again, can be added).

The seq command has more uses than the above description might suggest (in fact, it is mainly used for other things than just printing a sequence of numbers - after all, who would have a need to do that much). Here is one example, on Unix (from the Wikipedia article about seq):

# Remove file1 through file17:
for n in `seq 17`
do
    rm file$n
done

Note that those are backquotes or grave accents around seq 17 in the above code snippet. It uses sh / bash syntax, so requires one of them, or a compatible shell.

Here is the code for seq1.py:

'''
seq1.py
Purpose: To act somewhat like the Unix seq command.
Author: Vasudev Ram
Copyright 2017 Vasudev Ram
Web site: https://vasudevram.github.io
Blog: https://jugad2.blogspot.com
Product store: https://gumroad.com/vasudevram
'''

import sys

def main():
    sa, lsa = sys.argv, len(sys.argv)
    if lsa < 2:
        sys.exit(1)
    try:
        start = 1
        if lsa == 2:
            end = int(sa[1])
        elif lsa == 3:
            start = int(sa[1])
            end = int(sa[2])
        else: # lsa > 3
            sys.exit(1)
    except ValueError as ve:
        sys.exit(1)

    for num in xrange(start, end + 1):
        print num, 
    sys.exit(0)
    
if __name__ == '__main__':
    main()

And here are a few runs of seq1.py, and the output of each run, below:

$ py -2 seq1.py

$ py -2 seq1.py 1
1

$ py -2 seq1.py 2
1 2

$ py -2 seq1.py 3
1 2 3

$ py -2 seq1.py 1 1
1

$ py -2 seq1.py 1 2
1 2

$ py -2 seq1.py 1 3
1 2 3

$ py -2 seq1.py 4
1 2 3 4

$ py -2 seq1.py 1 4
1 2 3 4

$ py -2 seq1.py 2 2
2

$ py -2 seq1.py 5 3

$ py -2 seq1.py -6 -2
-6 -5 -4 -3 -2

$ py -2 seq1.py -4 -0
-4 -3 -2 -1 0

$ py -2 seq1.py -5 5
-5 -4 -3 -2 -1 0 1 2 3 4 5

There are many other possible uses for seq, if one uses one's imagination, such as rapidly generating various filenames or directory names, with numbers in them (as a prefix, suffix or in the middle), for testing or other purposes, etc.

- Enjoy.

- Vasudev Ram - Online Python training and consulting

Get updates (via Gumroad) on my forthcoming apps and content.

Jump to posts: Python * DLang * xtopdf

Subscribe to my blog by email

My ActiveState Code recipes

Follow me on: LinkedIn * Twitter

Managed WordPress Hosting by FlyWheel

Share |

Processing DSV data (Delimiter-Separated Values) with Python

By Vasudev Ram

I needed to process some DSV files recently. Here is a Python program I wrote for it, with a few changes over the original. E.g. I do not show the processing of the data here; I only read and print it. Also, I support two different command-line options (and ways) to specify the delimiter character.

DSV (Delimiter-separated values) is a common tabular text data format, with one record per line, and some number of fields per record, where the fields are separated or delimited by some specific character. Some common delimiter characters used in DSV files are tab (which makes them TSV files, Tab-Separated Values, common on Unix), comma (CSV files, Comma-Separated Values, a common spreadsheet and database import-export format), the pipe character (|), the colon (:) and others.

They - DSV files - are described in this section, Data File Metaformats, under Chapter 5: Textuality, of Eric Raymond (ESR)'s book, The Art of Unix Programming, which is a recommended read for anyone interested in Unix (one of the longest-lived operating systems [1]) and in software design and development.

[1] And, speaking a bit loosely, nowaday Unix is also the most widely used OS in the world, by a fair margin, due to its use (as a variant) in Android and iOS based mobile devices, both of which are Unix-based, not to mention Apple MacOS and Linux computers, which also are. Android devices alone number in the billions.

The program, read_dsv.py, is a command-line utility, written in Python, that allows you to specify the delimiter character in one of two ways:

- with a "-c delim_char" option, in which delim_char is an ASCII character,
- with a "-n delim_code" option, in which delim_code is an ASCII code.

It then reads either the files specified as command-line arguments after the -n or -c option, or if no files are given, it reads its standard input.

Here is the code for read_dsv.py:

from __future__ import print_function

"""
read_dsv.py
Author: Vasudev Ram
Web site: https://vasudevram.github.io
Blog: https://jugad2.blogspot.com
Product store: https://gumroad.com/vasudevram
Purpose: Shows how to read DSV data, i.e. 

https://en.wikipedia.org/wiki/Delimiter-separated_values 

from either files or standard input, split the fields of each
line on the delimiter, and process the fields in some way.
The delimiter character is configurable by the user and can
be specified as either a character or its ASCII code.

Reference:
TAOUP (The Art Of Unix Programming): Data File Metaformats:
http://www.catb.org/esr/writings/taoup/html/ch05s02.html
ASCII table: http://www.asciitable.com/
"""

import sys
import string

def err_write(message):
    sys.stderr.write(message)

def error_exit(message):
    err_write(message)
    sys.exit(1)

def usage(argv, verbose=False):
    usage1 = \
        "{}: read and process DSV (Delimiter-Separated-Values) data.\n".format(argv[0])
    usage2 = "Usage: python" + \
        " {} [ -c delim_char | -n delim_code ] [ dsv_file ] ...\n".format(argv[0])
    usage3 = [
        "where one of either the -c or -n option must be given,\n",  
        "delim_char is a single ASCII delimiter character, and\n", 
        "delim_code is a delimiter character's ASCII code.\n", 
        "Text lines will be read from specified DSV file(s) or\n", 
        "from standard input, split on the specified delimiter\n", 
        "specified by either the -c or -n option, processed, and\n", 
        "written to standard output.\n", 
    ]
    err_write(usage1)
    err_write(usage2)
    if verbose:
        for line in usage3:
            err_write(line)

def str_to_int(s):
    try:
        return int(s)
    except ValueError as ve:
        error_exit(repr(ve))

def valid_delimiter(delim_code):
    return not invalid_delimiter(delim_code)

def invalid_delimiter(delim_code):
    # Non-ASCII codes not allowed, i.e. codes outside
    # the range 0 to 255.
    if delim_code < 0 or delim_code > 255:
        return True
    # Also, don't allow some specific ASCII codes;
    # add more, if it turns out they are needed.
    if delim_code in (10, 13):
        return True
    return False

def read_dsv(dsv_fil, delim_char):
    for idx, lin in enumerate(dsv_fil):
        fields = lin.split(delim_char)
        assert len(fields) > 0
        # Knock off the newline at the end of the last field,
        # since it is the line terminator, not part of the field.
        if fields[-1][-1] == '\n':
            fields[-1] = fields[-1][:-1]
        # Treat a blank line as a line with one field,
        # an empty string (that is what split returns).
        print("Line", idx, "fields:")
        for idx2, field in enumerate(fields):
            print(str(idx2) + ":", "|" + field + "|")

def main():
    # Get and check validity of arguments.
    sa = sys.argv
    lsa = len(sa)
    if lsa == 1:
        usage(sa)
        sys.exit(0)
    if lsa == 2:
        # Allow the help option with any letter case.
        if sa[1].lower() in ("-h", "--help"):
            usage(sa, verbose=True)
            sys.exit(0)
        else:
            usage(sa)
            sys.exit(0)

    # If we reach here, lsa is >= 3.
    # Check for valid mandatory options (sic).
    if not sa[1] in ("-c", "-n"):
        usage(sa, verbose=True)
        sys.exit(0)

    # If -c option given ...
    if sa[1] == "-c":
        # If next token is not a single character ...
        if len(sa[2]) != 1:
            error_exit(
            "{}: Error: -c option needs a single character after it.".format(sa[0]))
        if not sa[2] in string.printable:
            error_exit(
            "{}: Error: -c option needs a printable ASCII character after it.".format(\
            sa[0]))
        delim_char = sa[2]
    # else if -n option given ...
    elif sa[1] == "-n":
        delim_code = str_to_int(sa[2])
        if invalid_delimiter(delim_code):
            error_exit(
            "{}: Error: invalid delimiter code {} given for -n option.".format(\
            sa[0], delim_code))
        delim_char = chr(delim_code)
    else:
        # Checking for what should not happen ... a bit of defensive programming here.
        error_exit("{}: Program error: neither -c nor -n option given.".format(sa[0]))

    try:
        # If no filenames given, read sys.stdin ...
        if lsa == 3:
            print("processing sys.stdin")
            dsv_fil = sys.stdin
            read_dsv(dsv_fil, delim_char)
            dsv_fil.close()
        # else (filenames given), read them ...
        else:
            for dsv_filename in sa[3:]:
                print("processing file:", dsv_filename)
                dsv_fil = open(dsv_filename, 'r')
                read_dsv(dsv_fil, delim_char)
                dsv_fil.close()
    except IOError as ioe:
        error_exit("{}: Error: {}".format(sa[0], repr(ioe)))
        
if __name__ == '__main__':
    main()

Here are test runs of the program (both valid and invalid), and the results of each one:

Run it without any arguments. Gives a brief usage message.

$ python read_dsv.py
read_dsv.py: read and process DSV (Delimiter-Separated-Values) data.
Usage: python read_dsv.py [ -c delim_char | -n delim_code ] [ dsv_file ] ...

Run it with a -h option (for help). Gives the verbose usage message.

$ python read_dsv.py -h
read_dsv.py: read and process DSV (Delimiter-Separated-Values) data.
Usage: python read_dsv.py [ -c delim_char | -n delim_code ] [ dsv_file ] ...
where one of either the -c or -n option must be given,
delim_char is a single ASCII delimiter character, and
delim_code is a delimiter character's ASCII code.
Text lines will be read from specified DSV file(s) or
from standard input, split on the specified delimiter
specified by either the -c or -n option, processed, and
written to standard output.

Run it with a -v option (invalid run). Gives the brief usage message.

$ python read_dsv.py -v
read_dsv.py: read and process DSV (Delimiter-Separated-Values) data.
Usage: python read_dsv.py [ -c delim_char | -n delim_code ] [ dsv_file ] ...

Run it with a -c option but no ASCII character argument (invalid run). Gives the brief usage message.

$ python read_dsv.py -c
read_dsv.py: read and process DSV (Delimiter-Separated-Values) data.
Usage: python read_dsv.py [ -c delim_char | -n delim_code ] [ dsv_file ] ...

Run it with a -c option followed by the pipe character (invalid run). The OS (here, Windows) gives an error message because the pipe character cannot be used to end a pipeline.

$ python read_dsv.py -c |
The syntax of the command is incorrect.

Run it with the -c option and the pipe character as the delimiter character, but protected (by double quotes) from interpretation by the OS shell (CMD).

$ python read_dsv.py -c "|" file1.dsv
processing file: file1.dsv
Line 0 fields:
0: |1|
1: |2|
2: |3|
3: |4|
4: |5|
5: |6|
6: |7|
Line 1 fields:
0: |field1|
1: |fld2|
2: | fld3 with spaces around it |
3: |    fld4 with leading spaces|
4: |fld5 with trailing spaces     |
5: |next field is empty|
6: ||
7: |last field|
Line 2 fields:
0: ||
1: |1|
2: |22|
3: |333|
4: |4444|
5: |55555|
6: |666666|
7: |7777777|
8: |88888888|
Line 3 fields:
0: ||
1: ||
2: ||
3: ||
4: ||
5: |                      |
6: ||
7: ||
8: ||
9: ||
10: ||

Run it with the -n option followed by 124, the ASCII code for the pipe character as the delimiter.

$ python read_dsv.py -n 124 file1.dsv
[Gives exact same output as the run above, as it should, because both use the same delimiter and read the same input file.]

Copy file1.dsv to file3.dsv. Change all the pipe characters (delimiters) to colons:
Run it with the -n option followed by 58, the ASCII code for the colon character.

$ python read_dsv.py -n 58 file3.dsv
[Gives exact same output as the run above, as it should, because other than the delimiters (pipe versus colon), the input is the same.]

I added support for the -n option to the program because it makes it more flexible, since you can specify any ASCII character as the delimiter (that makes sense), by giving its ASCII code.

And of course, to find out the values of the ASCII codes for these delimiter characters, I used the char_to_ascii_code.py program from my recent post:

Trapping KeyboardInterrupt and EOFError for program cleanup

You may have noticed that I mentioned delimiter characters and DSV files in that post too. The char_to_ascii_code.py utility shown in that post was created to find the ASCII code for any character (without having to look it up on the web each time).

- Enjoy.

- Vasudev Ram - Online Python training and consulting

- Black Flyday at Flywheel Wordpress Managed Hosting - get 3 months free on the annual plan.

Get updates on my software products / ebooks / courses.

Jump to posts: Python   DLang   xtopdf

Subscribe to my blog by email

My ActiveState recipes

Share |

Getting CPU info with D (the D language)

By Vasudev Ram

I have been using the D programming language for some time.

From the D home page:

[ D is a systems programming language with C-like syntax and static typing. It combines efficiency, control and modeling power with safety and programmer productivity. ]

[ Some information about D:

- D Language home page
- D Overview
- D on Wikipedia ]

Here is a D program, cpu_info.d, that gets some information about the CPU of your machine.

// cpu_info.d
// Author: Vasudev Ram - https://vasudevram.github.io 
// http://jugad2.blogspot.com
import std.stdio;
import core.cpuid;
void main()
{
    writeln("processor: ", processor());
    writeln("vendor: ", vendor());
    writeln("hyperThreading: ", hyperThreading());
    writeln("threadsPerCPU: ", threadsPerCPU());
    writeln("coresPerCPU: ", coresPerCPU());
}

The program can be compiled and run with:

$ dmd cpu_info.d

dmd is the D compiler (the name stands for Digital Mars D). It creates cpu_info.exe which I can then run:

$ cpu_info
processor: Intel(R) Core(TM) i3-2328M CPU @ 2.20GHz
vendor: GenuineIntel
hyperThreading: true
threadsPerCPU: 4
coresPerCPU: 2

- Vasudev Ram - Online Python training and consulting

Signup to hear about my new courses and products.

My Python posts     Subscribe to my blog by email

My ActiveState recipes

Share |

Microsoft supporting Ubuntu apps running on Windows

By Vasudev Ram


WINDOWS <-> UBUNTU

Seen today on HN:

Ubuntu on Windows (dustinkirkland.com)

(It's the top post on HN at the time I'm writing this, and for a while before.)

Original post here: Ubuntu on Windows -- The Ubuntu Userspace for Windows Developers by Dustin Kirkland of Canonical, the maker of Ubuntu.

I commented a few times and asked a few questions too.

It's a pretty interesting thread, IMO, for those with interest in the Windows and Linux operating systems.

There are a lot of technical topics discussed and also some business ones, related to this move. Senior people from the Linux and Windows camps participating.

E.g.:

[ > So do Cygwin and/or MSYS emulate the fork() system call

Yes. That's one thing we spent considerable engineering effort on in this first version of the Windows Subsystem for Linux: We implement fork in the Windows kernel, along with the other POSIX and Linux syscalls.
This allows us to build a very efficient fork() and expose it to the GNU/Ubuntu user-mode apps via the fork(syscall).
We'll be publishing more details on this very soon. ]

There was also discussion of the POSIX subsystem that was there on Windows for a few Windows versions (from NT). I had used it to run some of my Unix command-line utilities (that used mainly the stdio and stdlib C libraries [1]) on Windows, in the Windows NT and Windows 2000 days.

[1] Because the POSIX subsystem support on Windows was limited.

Here is another HN thread about it, at around the same time, though this one is off the front page now:

Microsoft and Canonical partner to bring Ubuntu to Windows 10 (zdnet.com)

- Vasudev Ram - Online Python training and programming

Signup to hear about new products and services I create.

Posts about Python  Posts about xtopdf

My ActiveState recipes

Share |

Generate Windows Task List to PDF with xtopdf

By Vasudev Ram

While working at the DOS command line in Windows, I had the idea of using the DOS TASKLIST command along with xtopdf, my PDF generation toolkit, to generate a list of currently running Windows tasks to a PDF file, along with some other info, such as whether a task is a service or a console process, the process id, the memory usage, etc. The TASKLIST command shows all that information, by default.

I also sorted the output in ascending order by the Mem Usage field, by passing it through the DOS SORT command. (I could have sorted it by any other field such as the Image Name or the PID, of course.) I starred out some of the fields in the output.

Here are the steps to generate a Windows task list as a PDF, using xtopdf:

( I use $ as the prompt, even in DOS :)

1: Run TASKLIST and redirect its output to a text file.

$ tasklist > tasklist.out

2: Sort the file into another file.

$ sort /+65 tasklist.out > tasklist.srt

(Sort the output of TASKLIST by the character position of the Mem Usage field.)

3: Go edit tasklist to put the header lines back at the top :)
[ They get dislodged by the sort. ]

[ This is not Unix, so you can't easily do the fast, fluid command-line data munging that you can on Unix, unless you use something like Cygwin or UWin.

UWin was developed by David Korn, creator of the Korn Shell, for Windows. You can get UWin from the AT&T site here (after doing a convoluted license agreement dance, last time I checked). But IMO, the dance is not too long, and is worth it, to get a suite of Unix tools that work well on Windows, and UWin is also smaller & lighter than Cygwin, though not so comprehensive.
Be sure to read the section "Korn shell and Microsoft" at the David Korn link above :-) ]

4: Pipe the sorted task list to StdinToPDF, to generate the PDF output.

$ type tasklist.srt | python StdinToPDF.py tasklist.pdf

We just pipe the output of TASKLIST to StdinToPDF.py (an xtopdf app), which can be used at the end of any arbitrary command pipeline that generates text (on Unix / Windows / Linux / Mac OS X), to convert that text to PDF.

A screenshot of the PDF output I got (viewed in Foxit PDF Reader), is shown at the top of this post.

- Enjoy.

- Vasudev Ram - Online Python training and programming

Signup to hear about new products and services I create.

Posts about Python  Posts about xtopdf

My ActiveState recipes

Share |

data_dump, a Python tool like Unix od (octal dump)

By Vasudev Ram

The Unix od command, which stands for octal dump, should be known to regular Unix users. Though the name includes the word octal (for historical reasons) [1], it supports other numeric systems as well; see below.

[1] See:

The Wikipedia page for od, which says that "od is one of the earliest Unix programs, having appeared in version 1 AT&T Unix."

od is a handy tool. It dumps the contents of a file (or standard input) to standard output, in "unambiguous" ways, such as the ability to show the file contents as numeric values (ASCII codes), interpreted as bytes / two-byte words / etc. It can do this in octal, decimal, binary or hexadecimal format. It can also show the content as characters. But the Unix cat command does that already, so the od command is more often used to show characters along with their numeric codes. It also shows the byte offset (from the start of the file) of every, say, 10th character in the file, in the left column of its output, so the user can keep track of where any content occurs in the file.

All this is useful because it allows Unix users (programmers and system administrators as well as end users) to inspect the contents of files in different ways (hex, binary, character, etc.). The files thus inspected could be text files or binary files of any kind. Often, programmers use the output of od to debug their application, by viewing a file that their program is either reading from or writing to, to verify that it contains what they expect, or to find that it contains something that they do not expect - which could be due either to invalid input or to a bug in their program causing incorrect output.

I needed to use od recently. Doing so made me think of writing a simple version of it in Python, for fun and practice. So I did it. I named it data_dump.py. Here is the code for it:

'''
Program name: data_dump.py
Author: Vasudev Ram.
Copyright 2015 Vasudev Ram.
Purpose: To dump the contents of a specified file or standard input, 
to the standard output, in one or more formats, such as:
    - as characters
    - as decimal numbers
    - as hexadecimal numbers
    - as octal numbers
    
Inspired by the od (octal dump) command of Unix, and intended to work,
very roughly, like it. Will not attempt to replicate od exactly or even 
closely. May diverge from od's way of doing things, as desired.
'''

# Imports:

from __future__ import print_function
import sys

# Global constants:

# Maximum number of character (from the input) to output per line.
MAX_CHARS_PER_LINE = 16

# Global variables:

# Functions:

def data_dump(infil, line_len=MAX_CHARS_PER_LINE, options=None):
    '''
    Dumps the data from the input source infil to the standard output.
    '''
    byte_addr = 0
    buf = infil.read(line_len)
    # While not EOF.
    while buf != '':
        # Print the offset of the first character to be output on this line.
        # The offset refers to the offset of that character in the input,
        # not in the output. The offset is 0-based.
        sys.stdout.write("{:>08s}: ".format(str(byte_addr)))

        # Print buf in character form, with . for control characters.
        # TODO: Change to use \n for line feed, \t for tab, etc., for 
        # those control characters which have unambiguous C escape 
        # sequences.
        byte_addr += len(buf)
        for c in buf:
            sys.stdout.write('  ') # Left padding before c as char.
            if (0 <= ord(c) <= 31) or (c == 127):
                sys.stdout.write('.')
            else:
                sys.stdout.write(c)
        sys.stdout.write('\n')

        # Now print buf in hex form.
        sys.stdout.write(' ' * 10) # Padding to match that of byte_addr above.
        for c in buf:
            sys.stdout.write(' ') # Left padding before c in hex.
            sys.stdout.write('{:>02s}'.format((hex(ord(c))[2:].upper())))
        sys.stdout.write('\n')
        buf = infil.read(line_len)
    infil.close()


def main():
    '''
    Checks the arguments, sets option flags, sets input source.
    Then calls data_dump() function with the input source and options.
    '''
    try:
        lsa = len(sys.argv)
        if lsa == 1:
            # Input from standard input.
            infil = sys.stdin
        elif lsa == 2:
            # Input from a file.
            infil = open(sys.argv[1], "rb")
        data_dump(infil)
        sys.exit(0)
    except IOError as ioe:
        print("Error: IOError: " + str(ioe))
        sys.exit(1)

if __name__ == '__main__':
    main()

And here is the output of a sample run, on a small text file:

$ data_dump.py t3
00000000:   T  h  e     q  u  i  c  k     b  r  o  w  n
           54 68 65 20 71 75 69 63 6B 20 62 72 6F 77 6E 20
00000016:   f  o  x     j  u  m  p  e  d     o  v  e  r
           66 6F 78 20 6A 75 6D 70 65 64 20 6F 76 65 72 20
00000032:   t  h  e     l  a  z  y     d  o  g  .  .  .  T
           74 68 65 20 6C 61 7A 79 20 64 6F 67 2E 0D 0A 54
00000048:   h  e     q  u  i  c  k     b  r  o  w  n     f
           68 65 20 71 75 69 63 6B 20 62 72 6F 77 6E 20 66
00000064:   o  x     j  u  m  p  e  d     o  v  e  r     t
           6F 78 20 6A 75 6D 70 65 64 20 6F 76 65 72 20 74
00000080:   h  e     l  a  z  y     d  o  g  .  .  .  T  h
           68 65 20 6C 61 7A 79 20 64 6F 67 2E 0D 0A 54 68
00000096:   e     q  u  i  c  k     b  r  o  w  n     f  o
           65 20 71 75 69 63 6B 20 62 72 6F 77 6E 20 66 6F
00000112:   x     j  u  m  p  e  d     o  v  e  r     t  h
           78 20 6A 75 6D 70 65 64 20 6F 76 65 72 20 74 68
00000128:   e     l  a  z  y     d  o  g  .
           65 20 6C 61 7A 79 20 64 6F 67 2E

$

Note that I currently replace control / non-printable characters by a dot, in the output. Another option could be to replace (at least some of) them with C escape sequences, such as \r (carriage return, ASCII 13), \n (line feed, ASCII 10), etc. That is the way the original od does it.

In a future post, I'll make some improvements, and also show and discuss some interesting and possibly anomalous results that I got when testing data_dump.py with different inputs.

Happy dumping! :)

Details of the above image are available here:

Truck image credits

- Vasudev Ram - Online Python training and programming

Signup to hear about new products and services I create.

Posts about Python  Posts about xtopdf

My ActiveState recipes

Share |

tabtospaces, utility to change tabs to spaces in Python files

By Vasudev Ram

Near the end of a recent blog post:

asciiflow.com: Draw flowcharts online, in ASCII

, I showed how this small snippet of Python code can be used to make a Python program usable as a component in a Unix pipeline:

for lin in sys.stdin:
    sys.stdout.write(process(lin))

Today I saw Raymond Hettinger (@raymondh)'s tweet about the -t and -tt command line options of Python:

#python tip: In Python 2, the -tt option raises an error when you foolishly mix spaces and tabs. In Python 3, that is always an error.

That made me think of writing a simple Python 2 tool to change the tabs in a Python file to spaces. Yes, I know it can be easily done in Unix or Windix [1] with any of sed / awk / tr etc. That's not the point. So here is tabtospaces.py:

import sys
for lin in sys.stdin:
    sys.stdout.write(lin.replace("\t", "    "))

[ Note: this code converts each tab into 4 spaces. It can be parameterized by passing a command-line option that specifies the number of spaces, such as 4 or 8, and then replacing each tab with that many spaces. Also note that I have not tested the program on many sets of data, just one for now. ]

I created a simple Python file, test1.py, that has mixed tabs and spaces to use as input to tabtospaces.py. Then I ran the following commands:

$ py -tt test1.py
  File "test1.py", line 4
    print arg,
              ^
TabError: inconsistent use of tabs and spaces in indentation

$ py tabtospaces.py < test1.py > test2.py

$ py -tt test2.py
0 1 2 3 4 5 6 7 8 9

which shows that tabtospaces.py does convert the tabs to spaces.

And you can see from this diff that the original test1.py and the test2.py generated by tabtospaces.py, differ only in the use of tabs vs. spaces:

$ fc /l test1.py test2.py
Comparing files test1.py and TEST2.PY
***** test1.py
    for arg in args:
                print arg,

***** TEST2.PY
    for arg in args:
        print arg,

*****

[1] Windix is the latest upcoming Unix-compatible OS from M$, due Real Soon Now. You heard it here first - TM.

- Vasudev Ram - Online Python training and programming

Dancing Bison Enterprises

Signup to hear about new software products or info-products that I create.

Posts about Python  Posts about xtopdf

Contact Page

Share |

A simple UNIX-like "which" command in Python

By Vasudev Ram

UNIX users are familiar with the which command. Given an argument called name, it checks the system PATH environment variable, to see whether that name exists (as a file) in any of the directories specified in the PATH. (The directories in the PATH are colon-separated on UNIX and semicolon-separated on Windows.)

I'd written a Windows-specific version of the which command some time ago, in C.

Today I decided to write a simple version of the which command in Python. In the spirit of YAGNI and incremental development, I tried to resist the temptation to add more features too early; but I did give in once and add the exit code stuff near the end :)

Here is the code for which.py:

from __future__ import print_function

# which.py
# A minimal version of the UNIX which utility, in Python.
# Author: Vasudev Ram - www.dancingbison.com
# Copyright 2015 Vasudev Ram - http://www.dancingbison.com

import sys
import os
import os.path
import stat

def usage():
    sys.stderr.write("Usage: python which.py name\n") 
    sys.stderr.write("or: which.py name\n") 

def which(name):
    found = 0 
    for path in os.getenv("PATH").split(os.path.pathsep):
        full_path = path + os.sep + name
        if os.path.exists(full_path):
            """
            if os.stat(full_path).st_mode & stat.S_IXUSR:
                found = 1
                print(full_path)
            """
            found = 1
            print(full_path)
    # Return a UNIX-style exit code so it can be checked by calling scripts.
    # Programming shortcut to toggle the value of found: 1 => 0, 0 => 1.
    sys.exit(1 - found)

def main():
    if len(sys.argv) != 2:
        usage()
        sys.exit(1)
    which(sys.argv[1])

if "__main__" == __name__:
        main()

And here are a few examples of using the command:

(Note: the tests are done on Windows, though the command prompt is a $ sign (UNIX default); I just set it to that because I like $'s and UNIX :)

$ which vim
\vim

$ which vim.exe
C:\Ch\bin\vim.exe

$ set PATH | grep -i vim73

$ addpath c:\vim\vim73

$ which.py vim.exe
C:\Ch\bin\vim.exe

c:\vim\vim73\vim.exe
$ which metapad.exe
C:\util\metapad.exe

$ which pscp.exe
C:\util\pscp.exe
C:\Ch\bin\pscp.exe

$ which dostounix.exe
C:\util\dostounix.exe

$ which pythonw.exe
C:\Python278\pythonw.exe
D:\Anaconda-2.1.0-64\pythonw.exe

# Which which is which? All four combinations:

$ which which
.\which

$ which.py which
.\which

$ which which.py
.\which.py

$ which.py which.py
.\which.py

As you can see, calling the which Python command with different arguments, gives various results, including sometimes finding one instance of vim.exe and sometimes two instances, depending on the values in the PATH variable (which I changed, using my addpath.bat script, to add the \vim\vim73 directory to it).

Also, it works when invoked either as which.py or just which.

I'll discuss my interpretation of these variations in an upcoming post - including a variation that uses os.stat(full_path).st_mode - see the commented part of the code under the line:

if os.path.exists(full_path):

Meanwhile, did you know that YAGNI was written about much before agile was a thing? IIRC, I've seen it described in either Kernighan and Ritchie (The C Programming Language) or in Kernighan and Pike (The UNIX Programming Environment). It could be possibly be older than that, say from the mainframe era.

Finally, as I was adding labels to this blog post, Blogger showed me "pywhich" as a label, after I typed "which" in the labels box. That reminded me that I had written another post earlier about a Python which utility (not by me), so I found it on my blog by typing in this URL:

http://jugad2.blogspot.in/search/label/pywhich

which finds all posts on my blog with the label 'pywhich' (and the same approach works for any other label); the resulting post is:

pywhich, like the Unix which tool, for Python modules.

- Enjoy.

- Vasudev Ram - Online Python training and programming

Dancing Bison Enterprises

Signup to hear about new products that I create.

Posts about Python  Posts about xtopdf

Contact Page

Share |

Nice post and discussion about Delphi

By Vasudev Ram

Saw this interesting post and subsequent Hacker News thread on Delphi:

Post:

Delphi, why won't it just die?

HN thread about the post.

- Vasudev Ram - Dancing Bison Enterprises

Contact Page

Share |

Windows msvcrt console I/O in Python

By Vasudev Ram

Python has some OS-specific modules (libraries), apart from its OS-independent ones.

There are OS-specific modules for Unix/Linux, Windows and Mac OS X - see the Python library documentation.

The msvcrt module of Python provides some Windows-specific routines.

This post shows one of the useful features of the msvcrt Python module.

# console_io.py 

# Using the Python msvcrt module's getch function
# to display key codes and characters.

# Author: Vasudev Ram - http://www.dancingbison.com
# Version: v0.1

from msvcrt import getch, getche

def main():
    print "This program prints keyboard characters"
    print "and their codes as keys are pressed."
    print "Press q (lower case) to quit."
    while True:
        print "Press a key: "
        c = getch()
        # Or use getche() to echo the characters as they are typed.
        print " c =", c
        print " ord(c) =", ord(c)
        if c == 'q':
            break

main()

Try running the above program with the command:

python console_io.py

after saving the program as console_io.py on Windows.

While the program is running, try pressing various keys on the keyboard, including the letter, digit, punctuation and other QWERTY keyboard keys, as well as the function keys (F1 to F10 or F12), other special keys like arrow keys and Ins / Del / Home / End / PgUp / PgDn, and see what happens.

If you don't already know why some of the behavior appears odd, try to figure it out by searching on the Web. Hint: Use terms like ASCII code, IBM PC key codes, scan codes as search keywords.

- Vasudev Ram - Dancing Bison Enterprises

Contact me

Share |

Python's os.startfile() can run or edit a file

By Vasudev Ram

The startfile() function in the os module of Python's standard library can be used to run or edit a file programmatically. (The documentation also mentions 'print' as another operation, but I didn't try it yet. It will probably print to the default configured Windows printer on your system.)

The startfile() function is available on Python for Windows, and gives Python a subset of the funtionality of the Windows start command, i.e. the "start filename" command that can be used at the Windows (DOS) command prompt. The start command is useful because it lets you start (i.e. usually open) a file without giving the name of the program needed to open it as a prefix to the filename, i.e. instead of typing "programname filename", where programname is the program needed to open filename, like Microsoft Word to open a Word document, you can just type "start filename", and Windows invokes the program associated with that filename (by looking in the registry, I guess).

Here is a program showing how to use os.startfile():

# test_startfile.py
import sys
import os

def test_startfile():
    filename = r'c:\temp\afile.bat'
    os.startfile(filename) # Default operation is 'run'
    #os.startfile(filename, 'edit') # Can specify 'edit' as the operation

test_startfile()

And here is the file afile.bat used in the above example:

@echo off
echo This is afile.bat
echo Current directory is %CD%
dir c:\/s/p

I put the recursive dir command in the batch file because, without it, the program works, but the output flashes off the screen before you can see it. Even if you use a statement like "a = raw_input()" (to make the program pause until you enter something) after the call to startfile(), the output still flashes off, because the batch file is executed in a subprocess, and that process terminates before raw_input() is called.

When the edit operation ran, it opened afile.bat in the editor that I had configured to edit text files, i.e. Metapad, which is a good lightweight substitute for the default Windows Notepad.

So os.startfile() can be useful to run or edit external programs under the control of your Python application on Windows.

P.S. At the Windows command prompt, you can also use start command to open another command window, like this:

start cmd

or just

start

I do that often when working in Windows, to open another command window, to do some work in it, typically in another directory, without needing to change to that directory in the original window (though you can use pushd and popd nowadays in Windows, like you could in Unix shells, from long back). Or sometimes to edit a program from one command window and test it from the other window.

You can also use the Windows start command like this:

start /wait command

which makes the current process wait until the started command exits.

- Vasudev Ram - Dancing Bison Enterprises

Share |

Strawberry Perl for Windows

By Vasudev Ram



Strawberry Perl is free a distribution of Perl for Windkws, that comes with many commonly used modules included, and also lets you download and use at least some Perl XS-based modules (Perl modules that require a C compiler, to be installed), since it comes with a basic setup of the MinGW C oompiler and related tools. (On Linux and most other UNIX variants, this is not a problem, since C compilers and the other needed tools are typically installed by default).

According to the entry for Strawberry Perl on Wikipedia, Larry Wall, original creator of Perl, has used Strawberry Perl and says it is a good port of Perl to Windows. There is a link to a ComputerWorld interview of Larry in which he says that. See page 3 of
that Larry Wall interview.

- Vasudev Ram - Dancing Bison Enterprises

Share |

Metapad is my text editor of choice for quick edits in Windows

By Vasudev Ram



Metapad is a great lightweight replacement for Windows Notepad.

Metapad is my preferred editor for light and quick editing tasks when on Windows. I even use it to read and edit programs sometimes.

There are, of course, more powerful editors for Windows, but Metapad is great for quick work, without having to set up a lot of options in the editor. When I am on a new Windows machine, I typically just download it, and only change the font to Courier New of a reasonable size, depending on the monitor resolution, and set tabs to 4 spaces, and I'm ready to go :)

I've been using Metapad for many years now.

Update: I should mention that Metapad does have many useful editing options that you can set, it's just that I don't use them most of the time.

But overall, it's a nice Windows software tool.

For bigger editing tasks, I use Vim / GVim, on Linux and Windows; it is one of the most powerful editors around, and I've been using it for years, and vi (the predecessor of vim), earlier.

Check out my vi quickstart tutorial if you are new to vi / Vim. It was published in Linux For You magazine, in 2003 or so. I first wrote it to help some colleagues who were familiar with Windows but not Unix, and who needed to be able to edit files on Unix. They told me that it helped them to quickly start using vi for basic editing tasks.

Interestingly, I just checked the Metapad side now, and came across these two items of interest:

Metapad turns 10

Metapad is now open sourced

- Vasudev Ram - Dancing Bison Enterprises

Share |

REBOL, language that influenced JSON, is now open source

Comments on: R3 Source Code Released!

REBOL is an interesting language. It's free to download, available for both Linux and Windows, and quite small in size (MB).

UPDATE: Carl's comment on building REBOL from source, in the REBOL repo on Github, mentions Android as a platform that REBOL can be built for. Interesting  ...

It can be used at the command line for useful one-liners, in command-line scripts, and even to write GUI programs.

It has built-in support for some common Internet protocols.

And many other features.

I had tried out REBOL  for some time, somewhat soon after it was first released several years ago, and found it fun to use.

REBOL  was created by Carl Sassenrath, who also was the main designer of the Amiga computer and OS, a very advanced PC for its time, including multitasking and advanced multimedia when almost no other computers had it.

Main REBOL site for downloading the language interpreters, documentation, examples:

www.rebol.com

REBOL is now open source:

https://github.com/rebol/r3

Hacker News thread about the open sourcing of REBOL:

http://news.ycombinator.com/item?id=4912963

Has interesting points. More than one commenter pointed out that REBOL was an influence on JSON, which was News (heh) to me:

https://erikeldridge.wordpress.com/2009/07/28/notes-bayjax-meetup-yahoo-sunnyvale-727-crockford-the-json-saga/

- Vasudev Ram
www.dancingbison.com

Clink adds bash-like features to Windows' cmd.exe

clink - Bringing Bash's powerful command line editing to Microsoft Windows' cmd.exe - Google Project Hosting

An interesting feature is that you can add Clink support to an already running cmd instance with its inject option.

- Vasudev Ram
www.dancingbison.com

psutil, Python tool to get process info and more

By Vasudev Ram

psutil is a versatile and cross-platform Python module for getting process and related information from the operating system.

Though the name psutil indicates that it works like the UNIX ps command (for process status), it does a lot more, and works on other platforms too.

IIRC, I had come across it earlier, but thought it was only for UNIX-like systems. I recently tried out psutil on Windows.

For certain Windows system processes, it may not give some info, due to those processes being privileged, according to one of the psutil authors, Giampaolo Rodola, who I emailed about it. He said: "On Windows you can't access all information about processes owned by NT AUTHORITY SYSTEM user. In those cases psutil will raise AccessDenied exception."

psutil can provide some information on:

- CPU time
- virtual memory
- swap memory (sic)
- disk partitions
- disk usage
- disk and network I/O
- users
- processes: name, executable name, working directory, command line, user name, user ids and group ids, threads, connections, file descriptors.

Here is a simple psutil test program which shows just a few features - output not well formatted, but readable. Try it on Windows or UNIX/Linux:

# test_psutil_00.py

import psutil

for pid in psutil.get_pid_list():
        p = psutil.Process(pid)
        p_name = p.name
        print "p_name:", p.name
        try:
                p_exe = p.exe
        except Exception, e:
                p_exe = "Can't access p.exe"
        print "p_exe:", p_exe
        try:
                p_cwd = p.getcwd()
        except Exception, e:
                p_cwd = "Can't access p.cwd()"
        print "p.getcwd():", p_cwd
        try:
                p_cmdline = p.cmdline
        except Exception, e:
                p_cmdline = "Can't access p.cmdline"
        print "p.cmdline:", p_cmdline

- Vasudev Ram

Pokki from SweetLabs, an app store for Windows (non-8)

By Vasudev Ram


Pokki, an app store for Windows

Seems like a good idea. I didn't like the idea of Windows 8 / Metro touch interface on PCs, the moment I read about it. I am much more productive with a traditional PC than with a touchscreen device like a smartphone. These people (SweetLabs) are doing something to showcase apps for traditional keyboard-and-mouse-based PCs. Early to say, but it could be a good opportunity for traditional Windows application developers to make money by highlighting their products in an app store like the mobile app stores.

VentureBeat article about Pokki; it says that it Pokki has "1 million monthly active users".

Excerpts:

[ Windows 8’s tablet interface forced on desktop users just ain’t going to cut it.
...
For Ng and co., Pokki is an opportunity to prove that Microsoft’s shift to tablet-driven innovation is premature.
...
Ng, however, embraces the keyboard and the mouse; he challenges the notion that PCs need to evolve to be more like touchscreen devices. His hope is that Pokki wins over the nine-to-fivers and the all-day computing types who want elegant and sophisticated applications that are tailored to fit their desktops, not their mobile devices.
...
Between 15 precent to 20 percent of active users of the top Pokki apps return on a daily basis, Ng said.
...
SweetLabs has a staff of more than 70 employees. The San Diego-based company is backed by Bessemer Venture Partners, Google Ventures, Intel Capital, and O’Reilly AlphaTech Ventures. It raised $13 million last September to finance Pokki’s development. ]

- Vasudev Ram - Dancing Bison Enterprises

Share |