Monday, March 21, 2011

Running one function while waiting for another to finish

 #  
 # this function does a cool spinny thing  
 #  
 def draw_ascii_spinner(delay=0.2):  
  for char in '/-\|': # there should be a backslash in here.  
   sys.stdout.write(char)  
   sys.stdout.flush()  
   time.sleep(delay)  
   sys.stdout.write('\r') # this should be backslash r.  
 #  
 # the below two functions   
 # do the work of backgrounding a process  
 # so we can print a progress indicator  
 #  
 def backGround(func,args,job):  
   job.append(func(*args))  
   
 def whileWeWait(func,arg):  
  job = []  
  t = threading.Thread(target=backGround, args=(func, (arg,), job))  
  t.start()  
  while t.is_alive():  
   draw_ascii_spinner()  
   t.join(0.2)  
   
  return job[0]  

## this is how we run it ..
## assuming we have a function named findCluster that needs a companyid
print whileWeWait(findCluster, companyid)
   

Monday, March 7, 2011

Printing a Grid with Python

I'm learning python and I wanted to print a grid (table) of data on the command line from a list

and here is what I ended up with ...


def printGrid(list):
  if list:
    # the character we use to demarc columns
    splitchar = '|'
    # how much padding 
    # (spaces to add to the data in each cell)
    pad = 1 
    # 2d dictionary with each row of data
    myArray = {}
    # the array that keeps count how wide the columns should be
    myColCount = {}
    # how many rows
    rownum = 0
    # how many columns
    colnum = 0
    # how wide the column
    colwidth = 0 


    # find out how many rows we have
    rownum = len(list)
    
    # find out how many columns we have 
    for cols in list[0].split(splitchar):
      # we also initialize the values of 
      # myColCount for each column and 
      # we set it to 0
      myColCount[colnum] = 0
      # increase the colnum value by 1
      colnum += 1 
    
    # for each row ...
    for r in range(rownum):
      # we populate myArray dictionary
      # with a list created by splitting
      # on a defined character (default is |)
      myArray[r] = list[r].split('|')
    
    #loop through columns
    for c in range(colnum):
      #then loop through rows
      for r in range(rownum):
        length = len(myArray[r][c])

        #if the length of the string is bigger or equal 
        # than what is in the myColCount dictionary ...
        if length >= myColCount[c]:
          # we set the new value to length + padding
          # and that is how we populate the myColCount dictionary
          # with column width values
          myColCount[c] = (length + pad)
          colwidth = myColCount[c]

    # for each row of data ...
    for r in range(rownum):
      # we reset str and border on each iteration of this outer loop
      str = ''
      border = ''
      # for each column ...
      for c in range(colnum):
        #all our column widths are stored in the myColCount dictionary
        colwidth = myColCount[c]
        # we set the border by multiplying dashes by the column width
        border += '+' + "%-*s" % (colwidth,'-'*(colwidth))
        # we set the string with a pipe character and set it 
        # to the width specified by colwidth and we use the - to left align
        str += "|" + "%-*s" % (colwidth, myArray[r][c])

      # we add an ending mark and newline
      border += "+\n"
      # we add an ending mark and newline
      str += "|\n"
      print border
      print str
    # we print the bottom border    
    print border

Tuesday, October 19, 2010

Animated Gifs with Linux, Gimp, mPlayer

I have learned to create animated GIFs from YouTube clips in Linux and this is how I did it.

With FireFox I went to the desired video on YouTube and played the video I wanted.

Looked for the latest file created in /tmp
 
ls -lrt /tmp/Flash*
/tmp/FlashXXzPQB05

Copied the latest Flash* file created to my homedir

cp /tmp/FlashXXzPQB05 /my/home/dir/SomeFile.flv

I used the following command several times, each time adjusting the start time and stop time until I got the section of the video I wanted

mplayer -ao null -loop 0 -ss 00:00:03 -endpos 8 SomeFile.flv

When I got the section of video I wanted, I then converted the video to a series of JPEGs that got dumped to the AnimGifs directory

mplayer -ao null -ss 00:00:03 -endpos 8 FreshPrince.flv -vo jpeg:outdir=AnimGifs

  1. Open the first image with GIMP (I'm using 2.6) 
  2. Open the rest of the images by going to File->Open as Layers and selecting all the desired images (except the first one) 
  3. Save the image as SomeFile.gif and choose the save as animation option and click export 

If the image is too fast .. run the whole procedure again but save the GIF with 45 to 40 MilliSecond delay.

Wednesday, December 9, 2009

Space Alllocation Web Script for EMC LUNs

This is a JQuery / JQuery UI script I put together to calculate how much space to add to a filesystem based on percentage used.

Here is the HTML form

1:  <div id="formBox"> 
2: <form id="calcQuery" >
3: <fieldset>
4: <legend>File System Space Calculator</legend>
5: <p>
6: <label for="curr_alloc">Current Space Allocation:</label>
7: <br />
8: <input type="text" size="5" name="curr_alloc" id="curr_alloc" />
9: &nbsp;KB <input type="radio" name="curr_unit" value="KB" />
10: &nbsp;MB <input type="radio" name="curr_unit" value="MB" />
11: &nbsp;GB <input type="radio" name="curr_unit" value="GB" checked/>
12: &nbsp;TB <input type="radio" name="curr_unit" value="TB" />
13: </p>
14: <p>
15: <label for="curr_percent">Current Usage Percentage:</label>
16: <br />
17: <input type="text" size="5" name="curr_percent" id="curr_percent" />
18: </p>
19: <p>
20: <label for="desired_percent">Desired Usage Percentage:</label>
21: <br />
22: <input type="text" size="5" name="desired_percent" id="desired_percent" />
23: </p>
24: <br />
25: <p>
26: <!-- input type="submit" value="calculate"/ --></p>
27: </fieldset>
28: </form>
29: <div id="calcBox"> </div>
30: </div>


here is the javascript for that pops up a dialog with the above form

1:  function clearForm(form) 
2: {
3: $(":input", form).each(function()
4: {
5: var type = this.type;
6: var tag = this.tagName.toLowerCase();
7: if (type == 'text')
8: this.value = "";
9: else if (type == 'radio')
10: $('input[name=curr_unit][value=GB]').attr('checked', true);
11: });
12: };
13: // form popup
14: $(document).ready(function()
15: {
16: $("#formBox").dialog({
17: bgiframe: true,
18: autoOpen: false,
19: height: 600,
20: width: 400,
21: modal: false,
22: closeOnEscape: true,
23: close: function()
24: {
25: $("#calcBox").hide();
26: clearForm("#calcQuery");
27: },
28: title: "Calculator",
29: buttons: {
30: "Calculate": function() {
31: // form post
32: $.ajax({
33: type: "POST",
34: url: "calc.php",
35: data: $("#calcQuery").serialize(),
36: dataType: "html",
37: success: function(response)
38: {
39: $("#calcBox").html(response);
40: $("#calcBox").show();
41: clearForm("#calcQuery");
42: },
43: error: function
44: (xhr, ajaxOptions, thrownError)
45: {
46: alert(xhr.status);
47: alert(thrownError);
48: }
49: }).responseText;
50: // form post
51: }
52: }
53: });
54: $('#calcButton').click(function(){
55: $('#formBox').dialog('open');
56: return false;
57: });
58: });


and here is the PHP code that does the actual calculations ...

1:  <?php 
2: function KBtoGB($a)
3: {
4: /*
5: * Convert KiloBytes to GigaBytes
6: * and round up (if needed)
7: */
8: $b = ($a / 1024) / 1024;
9: return ceil($b);
10: }
11: function MBtoGB($a)
12: {
13: /*
14: * Convert MegaBytes to GigaBytes
15: * and round up (if needed)
16: */
17: $b = $a / 1024;
18: return ceil($b);
19: }
20: function TBtoGB($a)
21: {
22: /*
23: * Convert TeraBytes to GigaBytes
24: * and round up (if needed)
25: */
26: $b = $a * 1024;
27: return ceil($b);
28: }
29: function altUnit($fmeasure, $umeasure)
30: {
31: /*
32: * This function gives
33: * alternative file system size units of measure
34: * $fmeasure is the filesystem size
35: * $umeasure is the unit of measure used
36: *
37: * round($somenum, 2) rounds to the nearest two decimal places
38: *
39: */
40: switch ($umeasure)
41: {
42: case "KB":
43: if ($fmeasure >= 1024)
44: {
45: $h = $fmeasure /1024;
46: $i = " or ".round($h, 2)."MB";
47: if ($h >= 1024)
48: {
49: $j = $h /1024;
50: $i .= " or ".round($j, 2)."GB\n";
51: if ($j >= 1024)
52: {
53: $k = $j /1024;
54: $i .= " or ".round($k, 2)."TB\n";
55: return $i;
56: }
57: return $i;
58: }
59: return $i;
60: }
61: break;
62: case "MB":
63: if ($fmeasure >= 1024)
64: {
65: $h = $fmeasure /1024;
66: $i = " or ".round($h, 2)."GB";
67: if ($h >= 1024)
68: {
69: $j = $h /1024;
70: $i .= " or ".round($j, 2)."TB\n";
71: return $i;
72: }
73: return $i;
74: }
75: break;
76: case "GB":
77: if ($fmeasure >= 1024)
78: {
79: $h = $fmeasure /1024;
80: $i = " or ".round($h, 2)."TB";
81: return $i;
82: }
83: break;
84: case "TB":
85: $h = $fmeasure * 1024;
86: if ($h >= 1024)
87: {
88: $i = " or ".round($h, 2)."GB";
89: return $i;
90: }
91: break;
92: }
93: }
94: function calcLun($lsize, $fsize)
95: {
96: /*
97: * Calculate the number of LUNs needed
98: * according to the file system size
99: * the final number is then rounded up (if needed)
100: *
101: * lsize = size of LUN
102: * fsisze = filesystem size
103: * lnum = number of luns
104: */
105: $lnum = $fsize / $lsize;
106: return ceil($lnum);
107: }
108: function calcSpace($cur_alloc, $unit, $cur_percent, $des_percent)
109: {
110: // take the raw numbers and do the raw math
111: $a = (($cur_alloc * $cur_percent) / $des_percent) - $cur_alloc;
112: /*
113: * we do different stuff
114: * depending on what
115: * unit of measure was clicked in the form
116: *
117: * the KBtoGB, MBtoGB, TBtoGB and calcLuns functions
118: * are used below
119: *
120: * In case you don't know the ceil() function
121: * is a PHP built in function
122: * it takes a decimal number and rounds up
123: */
124: switch ($unit)
125: {
126: case "KB":
127: $b = KBtoGB($a);
128: $c = calcLun(33,$b);
129: $d = calcLun(72,$b);
130: $e = '(('.ceil($a).$unit.' / 1024) / 1024) / 33 = '.$c.' LUNs Required';
131: $f = '(('.ceil($a).$unit.' / 1024) / 1024) / 72 = '.$d.' LUNs Required';
132: break;
133: case "MB":
134: $b = MBtoGB($a);
135: $c = calcLun(33,$b);
136: $d = calcLun(72,$b);
137: $e = '( '.ceil($a).$unit.' / 1024 ) / 33 = '.$c.' LUNs Required';
138: $f = '( '.ceil($a).$unit.' / 1024 ) / 72 = '.$d.' LUNs Required';
139: break;
140: case "GB":
141: $c = calcLun(33,$a);
142: $d = calcLun(72,$a);
143: $e = ceil($a).' / 33 = '.$c.' LUNs Required';
144: $f = ceil($a).' / 72 = '.$d.' LUNs Required';
145: break;
146: case "TB":
147: $b = TBtoGB($a);
148: $c = calcLun(33,$b);
149: $d = calcLun(72,$b);
150: $e = '('.ceil($a).$unit.' * 1024) / 33 = '.$c.' LUNs Required';
151: $f = '('.ceil($a).$unit.' * 1024) / 72 = '.$d.' LUNs Required';
152: break;
153: }
154: $g = '(('.$cur_alloc.$unit.' * '.$cur_percent.'%) / '.$des_percent.'%) - '.$cur_alloc.$unit;
155: $g .= ' = '.$a.$unit." which rounds up to ".ceil($a).$unit;
156: //call the altUnit function
157: $i = altUnit($a,$unit);
158: /*
159: * if there are alternative Units to print out
160: * example: 1024 GB or 1 TB (TB is the alternative unit)
161: */
162: if ($i)
163: {
164: $z = "<br /><p>you need about ".ceil($a).$unit.'&nbsp;'.$i."</p>";
165: }
166: else
167: {
168: $z = "<br /><p>you need about ".ceil($a).$unit."</p>";
169: }
170: //end of if
171: $z .= "<br /><p>That would be :<br />";
172: $z .= "\t".ceil($c)." 33GB LUNs <br />";
173: $z .= "&nbsp;or<br />\n";
174: $z .= "\t".ceil($d)." 72GB LUNs <br />";
175: $z .= "</p>";
176: $z .= "<br /><hr /><br />\n";
177: $z .= $g.'<br /><br />&nbsp; For 33GB LUNs '.$e.'<br />&nbsp; For 72GB LUNs '.$f."\n";
178: $z .= '<br /><br /><table border="0"><tr style="border-bottom: solid 2px #000;>';
179: $z .= '<td rowspan="2" valign="center" align="center">Formula: </td>&nbsp;';
180: $z .= '<th style="border-bottom: solid 2px #000; valign="bottom" align="center">Current Allocation x Current Percentage Used</th>';
181: $z .= '<th valign="center" align="center" rowspan="2">&nbsp; - Current Allocation</tr>';
182: $z .= '<tr><td>&nbsp;</td><th>Desired Percentage</th><td>&nbsp;</td></tr></table>';
183: print "$z";
184: }
185: if ($_POST['curr_alloc'] and $_POST['curr_unit'] and $_POST['curr_percent'] and $_POST['desired_percent'])
186: {
187: calcSpace($_POST['curr_alloc'], $_POST['curr_unit'], $_POST['curr_percent'], $_POST['desired_percent']);
188: }
189: else
190: {
191: print '<h1 style="font-weight: bold; color: #A22903;">Something is missing in the form</h1>';print "\n";
192: }
193: ?>

Friday, March 6, 2009

Fine Grained Auditing for Oracle XE

Oracle XE does not have FGA (Fine Grained Auditing).
So I had to implement a package and a trigger in order to keep a fine grained record of what data and what column and on what table was the data added,deleted or inserted.
I received quite a bit of help from the Oracle Forum.
I have created a table called "audit_logs" to store the records of inserts,deletes and updates.




Below is the SQL structure for the "audit_logs" table.
CREATE TABLE  "AUDIT_LOGS" 
( "TABLENAME" VARCHAR2(4000),
"COLUMNNAME" VARCHAR2(4000),
"USERNAME" VARCHAR2(4000),
"OLDVAL" VARCHAR2(4000),
"NEWVAL" VARCHAR2(4000),
"AUDITDATE" TIMESTAMP (6)
)
/


Below is the code for the "auditpkg" package.


create or replace package auditpkg AS -- package spec
procedure auditproc( p_tablename varchar2, p_columnname varchar2,
p_oldval varchar2, p_newval varchar2 );
procedure auditproc( p_tablename varchar2, p_columnname varchar2,
p_oldval number, p_newval number );
procedure auditproc( p_tablename varchar2, p_columnname varchar2,
p_oldval date, p_newval date );
end auditpkg;

/

SHOW ERRORS

create or replace package body auditpkg AS -- package body
procedure auditproc
( p_tablename varchar2, p_columnname varchar2, p_oldval varchar2, p_newval varchar2 )
IS
BEGIN
if (p_oldval is null and p_newval is not null)
or (p_oldval is not null and p_newval is null)
or (p_oldval != p_newval)
then
insert into audit_logs( tablename, columnname, username, oldval, newval, auditdate )
values (p_tablename, p_columnname, USER, p_oldval, p_newval, SYSDATE );
end if;
END auditproc;

procedure auditproc
( p_tablename varchar2, p_columnname varchar2, p_oldval number, p_newval number )
IS
BEGIN
if (p_oldval is null and p_newval is not null)
or (p_oldval is not null and p_newval is null)
or (p_oldval != p_newval)
then
insert into audit_logs( tablename, columnname, username, oldval, newval, auditdate )
values (p_tablename, p_columnname, USER, p_oldval, p_newval, SYSDATE );
end if;
END auditproc;

procedure auditproc
( p_tablename varchar2, p_columnname varchar2, p_oldval date, p_newval date )
IS
BEGIN
if (p_oldval is null and p_newval is not null)
or (p_oldval is not null and p_newval is null)
or (p_oldval != p_newval)
then
insert into audit_logs( tablename, columnname, username, oldval, newval, auditdate )
values (p_tablename, p_columnname, USER, p_oldval, p_newval, SYSDATE );
end if;
END auditproc;

END auditpkg;

/
SHOW ERRORS



Below is the code for the "audit_test_audit_trig" trigger
There must be one trigger for each table, each with its own unique trigger name of course.
In the case below, the trigger is for a table called "audit_test"
create or replace trigger audit_test_audit_trig
after insert or update or delete on audit_test
for each row
begin
auditpkg.auditproc( 'AUDIT_TEST', 'COL1', :old.col1, :new.col1 );
auditpkg.auditproc( 'AUDIT_TEST', 'COL2', :old.col2, :new.col2 );
auditpkg.auditproc( 'AUDIT_TEST', 'COL3', :old.col3, :new.col3 );
end;

Saturday, September 20, 2008

WMA to MP3 Script for Linux

I ran into a Rap artist who has his files as *.wma files for download.
So I created a bash/zenity script to convert those files to *.mp3s
#!/bin/bash
choice="`zenity --file-selection --directory --title="Select the directory with your wma files"`"
artist="`zenity --entry --title="Artist Name" --text="Enter Artist Name"`"
genre="`zenity --entry --title="Genre" --text="Enter Genre" --entry-text="Rap"`"
album="`zenity --entry --title="Album Name" --text="Enter Album Name"`"
year="$(zenity --entry --title="Album Year" --text="Enter Album Year" --entry-text="`date +%Y`")"


case $? in
0)
cd "$choice"
if [ $PWD = "$choice" ]; then
if [ ! -d 'workdir' ]; then
mkdir workdir
fi
if [ ! -d 'mp3s' ]; then
mkdir mp3s
fi
if [ ! -d 'wavs' ]; then
mkdir wavs
fi
zenity --info --text="This will take some time. Please be patient."

for file in *.wma
do cp "$file" ./workdir/`echo $file | sed -e 's/ */_/g' -e 's/[-]/_/g' -e 's/[.][.]//'`
done

cd workdir
for j in *.wma
do mplayer -quiet -vo null -vc dummy -af resample=44100 -ao pcm:waveheader:file=../wavs/`basename $j .wma`.wav $j &> /dev/null
done
for k in ../wavs/*.wav
do lame -b 192 $k ../mp3s/`basename $k .wav`.mp3 2>&1 | (zenity --progress --auto-close --pulsate --percentage=0 --text=`basename $k .wav`)
done
curdir=`basename $PWD`
if [ $curdir = 'workdir' ]; then

cd ../
count=`ls -1 mp3s/*mp3|wc -l`
#rm -fR workdir
#rm -fR wavs
cd mp3s
for l in *.mp3;do mp3info -a "$artist" -g "$genre" -l "$album" -y "$year" -n "$(echo $l|awk -F'_' '{print $1}')" -t "$(echo $l|sed -e 's/.._//' -e 's/.mp3//g' -e 's/[_]/ /g')" $l ;done
zenity --info --text=$count" files converted"

fi

cd
fi


;;
1)
zenity --error --text "No Directory Selected"
;;
-1)
zenity --error --text "No Directory Selected"
;;
esac






Needed packages:
  • zenity
  • mplayer (with windows codecs/etc)
  • lame
  • mp3info

Monday, March 3, 2008

Dell Vostro 1500 Triple Boot XP Media Direct

I just got a new Dell Vostro Here are the specs:
  • Vostro 1500, Intel Core 2 Duo T5470, 1.6GHz, 800Mhz FSB, 2M L2 Cache
  • 15.4 inch Wide Screen XGA LCD display with TrueLife for Vostro 1500
  • 2GB, DDR2, 667MHz 2 DIMM
  • 128MB NVIDIA GeForce 8400M GS (specifically chose this for better Linux compat)
  • 250GB Hard Drive, 5400RPM, SA160, for Dell Vostro 1500 Notebook
  • Genuine Windows XP Home Edition, SP2 (either this or Vista YUCK!)
  • Integrated 10/100 Network Cardand Modem, for Inspiron (Detected from Installer)
  • 8X DVD+/-RW with double-layer DVD+R write capability
  • Intel 4965AGN Wireless-N Mini card (specifically chosen for Linux Compat, we shall see)
  • No Built in Camera 1500 (trust me , you don't wanna see my ugly mug)
  • Dell Wireless 355 BlueTooth Module, (2.0+EDR) (have no idea why I chose this, besides cool factor)
  • Factory Enable 20GB Primary Partition (just to make it easier fer me)
I blew away the Windows XP and Dell Media Direct partition, but then I decided I wanted to play
with Dell Media Direct.
Sadly Dell Media Direct requires Windows XP to be installed.

So I booted with the Dell Media Direct disc. It gave me the option of using all the space on the harddisk or specify a size of the c:\ drive. I used the "use entire disk" option.
When the Dell Media Direct finished its process, it requested the XP install disc.
I ejected the Dell Media Direct disc and inserted the XP install disc and pressed enter.
After the looooong XP install, I put the Dell Media Disc back in so it can finish installing.

After the install I shutdown the machine and pressed the "Home" button and Dell Direct Media did its first time config and all was set. I exited out of Dell Direct Media and booted into Windows.

I installed Partition Magic (I was too lazy to download a Knoppix or Gnoppix disc image)
I shrunk the c: drive to a mere 20 GigaBytes. I rebooted and upon reboot Partition Magic did the drive shrink operation. After Partition Magic finished , I booted into Windows XP and all seemed well. I shutdown the machine and pressed the "Home" button and Dell media Direct worked.

I turned on the laptop and pressed 'F2" to get into the BIOS (this gave me time to insert the Ubuntu CD). I exited out of the BIOS and the machine booted with the Ubuntu Studio CD.

Network:
The installer did not detect my Intel Wireless Network card, so I connected the machine via ethernet to my network. It grabbed an IP off of my DHCP server and the install proceeded.

Disk Partition:
This is my partition scheme:
  • /dev/sda1 FAT16 47 MB for Dell Utility
  • /dev/sda2 NTFS 20 GB for Windows XP
  • /dev/sda4 EXT3 100 MB for /boot for Ubuntu Studio
  • /dev/sda3 extended
  • /dev/sda7 LVM 60 GB for Ubuntu Studio
  • /dev/sda5 FAT32 for Dell Media Direct (I suspect)
  • /dev/sda6 NTFS for Shared Data (the rest of the space of the drive)

Logical Volumes:

LV VG Attr LSize
homelv rootvg -wi-ao 4.00G
rootlv rootvg -wi-ao 10.00G
swaplv rootvg -wi-ao 4.00G
I am using logical volumes to I can move space around as needed.

The install went well and these are the minor issues I faced after installing:

NVIDIA Card:
By Default Ubuntu uses the opensource driver.
I enabled the NVidia Driver by going to:
System Menu then -->Administration-->Restricted Drivers Manager
I enabled the NVidia Driver it did its thing and I rebooted and all was well.

Sound Issue:
Sound was not working so I had to do :
sudo apt-get install linux-backports-modules-rt
I rebooted and sound worked

Wireless Network:
The Wireless Device was recognized from first boot.
I installed kwlan and was able to setup wireless networking.
Good thing I chose the Intel Wireless Device.

Extra Software:
I installed extra software by first installing AutoMatix

Obligatory Screenshot