Swig is broken in Ubuntu 16.04 (xenial)

Posted on February 11, 2017 by Nick

If you’re building a swig Python module and encounter errors like:

AttributeError: 'Foobar' object has no attribute 'this'

1	AttributeError: 'Foobar' object has no attribute 'this'

where Foobar is some class that is being wrapped, then you are encountering a swig-3.08 bug. Unfortunately this is still the official package in Ubuntu 16.04. You can vote for the issue here.

Until the package is updated, one can download and manually install an updated swig package by following the following recipe:

# install prerequisite 
sudo apt-get install libpcre3-dev

# download swig 3.0.12
wget -O swig-3.0.12.tar.gz https://downloads.sourceforge.net/project/swig/swig/swig-3.0.12/swig-3.0.12.tar.gz?r=https%3A%2F%2Fsourceforge.net%2Fprojects%2Fswig%2Ffiles%2Fswig%2Fswig-3.0.12%2Fswig-3.0.12.tar.gz%2Fdownload&ts=1486782132&use_mirror=superb-sea2

# extract and configure
tar xf swig-3.0.12.tar.gz
cd swig-3.0.12
./configure --prefix=/usr

# build
make -j 4

# install
sudo make install

# check version reported
swig -version

SWIG Version 3.0.12

Compiled with g++ [x86_64-pc-linux-gnu]

Configured options: +pcre

Please see http://www.swig.org for reporting bugs and further information

# install prerequisite

sudo apt-get install libpcre3-dev

# download swig 3.0.12

wget -O swig-3.0.12.tar.gz https://downloads.sourceforge.net/project/swig/swig/swig-3.0.12/swig-3.0.12.tar.gz?r=https%3A%2F%2Fsourceforge.net%2Fprojects%2Fswig%2Ffiles%2Fswig%2Fswig-3.0.12%2Fswig-3.0.12.tar.gz%2Fdownload&ts=1486782132&use_mirror=superb-sea2

# extract and configure

tar xf swig-3.0.12.tar.gz

cd swig-3.0.12

./configure --prefix=/usr

# build

make -j 4

# install

sudo make install

# check version reported

swig -version

SWIG Version 3.0.12

Compiled with g++ [x86_64-pc-linux-gnu]

Configured options: +pcre

Please see http://www.swig.org for reporting bugs and further information

Posted in computing, programming | Leave a comment

Building jEdit from scratch

Posted on February 7, 2017 by Nick

I’ve been using jEdit for decades as it provides a nice text editor with similar feature set as something I used at Raytheon called SlickEdit:

easily records a macro of all actions in the editor
easily playback the recorded macro
save the macro and map to keyboard shortcut
vertical selection

But lately some of my favorite plugins have stop working due to an old plugin that hasn’t been recompiled. So I’ll try to document the process of checking out the jEdit code to recompile it from scratch and recompile the plugins I depend on.

If I have time, I might make a Docker image to do this so it’s easily repeatable for others too.

Getting the jEdit code

To checkout the jEdit code do:

svn co http://svn.code.sf.net/p/jedit/svn/jEdit/trunk jedit_ws
# ...
Checked out revision 24599

svn co http://svn.code.sf.net/p/jedit/svn/jEdit/trunk jedit_ws

# ...

Checked out revision 24599

Building jEdit

Now that we have the latest trunk code do:

# install the ant builder tool if necessary
sudo apt-get install ant
cd jedit_ws
ant
# ...
BUILD FAILED
/home/nhilton/development/jedit/jedit-trunk/build.xml:268: Unable to find a javac compiler;
com.sun.tools.javac.Main is not on the classpath.
Perhaps JAVA_HOME does not point to the JDK.
It is currently set to "/usr/lib/jvm/java-8-openjdk-amd64/jre"

# install the ant builder tool if necessary

sudo apt-get install ant

cd jedit_ws

ant

# ...

BUILD FAILED

/home/nhilton/development/jedit/jedit-trunk/build.xml:268: Unable to find a javac compiler;

com.sun.tools.javac.Main is not on the classpath.

Perhaps JAVA_HOME does not point to the JDK.

It is currently set to "/usr/lib/jvm/java-8-openjdk-amd64/jre"

Oops, I guess I need a java compiler. I tried to install openjdk-9-jdk but got an error. Stackoverflow suggested the command below.

udo apt-get -o Dpkg::Options::="--force-overwrite" install openjdk-9-jdk
#...
ant
Buildfile: /home/nhilton/development/jedit/jedit-trunk/build.xml
init:
 [echo] Buildfile for jedit (/home/nhilton/development/jedit/jedit-trunk/build.xml)
check-ivy:
download-ivy:
init-ivy:
BUILD FAILED
/home/nhilton/development/jedit/jedit-trunk/build.xml:111: Unable to load a script
engine manager (org.apache.bsf.BSFManager or javax.script.ScriptEngineManager)

udo apt-get -o Dpkg::Options::="--force-overwrite" install openjdk-9-jdk

#...

ant

Buildfile: /home/nhilton/development/jedit/jedit-trunk/build.xml

init:

[echo] Buildfile for jedit (/home/nhilton/development/jedit/jedit-trunk/build.xml)

check-ivy:

download-ivy:

init-ivy:

BUILD FAILED

/home/nhilton/development/jedit/jedit-trunk/build.xml:111: Unable to load a script

engine manager (org.apache.bsf.BSFManager or javax.script.ScriptEngineManager)

This problem turned out to be a bug in apache ant 1.9.6.

Upgrading Apache Ant on Ubuntu 16.04

I couldn’t find an existing .deb package so I installed the binary from ant.apache.org. I used the following recipe:

# download
wget http://www-us.apache.org/dist//ant/binaries/apache-ant-1.10.1-bin.tar.gz

# untar into /usr/local
sudo tar xf apache-ant-1.10.1-bin.tar.gz -C /usr/local

# soft link
sudo ln -s /usr/local/apache-ant-1.10.1 /usr/local/ant

# export these in your env
export PATH=/usr/local/ant/bin:$PATH
export ANT_HOME=/usr/local/ant

# download

wget http://www-us.apache.org/dist//ant/binaries/apache-ant-1.10.1-bin.tar.gz

# untar into /usr/local

sudo tar xf apache-ant-1.10.1-bin.tar.gz -C /usr/local

# soft link

sudo ln -s /usr/local/apache-ant-1.10.1 /usr/local/ant

# export these in your env

export PATH=/usr/local/ant/bin:$PATH

export ANT_HOME=/usr/local/ant

Now jedit builds!

ant
# ...
build:
      [jar] Building jar: /home/nhilton/development/jedit/jedit-trunk/build/jedit.jar

BUILD SUCCESSFUL
Total time: 15 seconds

ant

# ...

build:

[jar] Building jar: /home/nhilton/development/jedit/jedit-trunk/build/jedit.jar

BUILD SUCCESSFUL

Total time: 15 seconds

Posted in computing | 1 Comment

P5js player widget in pure javascript

Posted on January 10, 2017 by Nick

Okay, so I got really excited about p5js, checkout their examples page. But I struggled a little bit to include a sketch on my wordpress blog, the canvas was showing up at the bottom of the website as I discussed in my previous post.

I got some help from the folks at p5js and they pointed out their embedding page. While there I found the p5js-widget tool. It was very close to what I wanted, but since I’m about to do more sketches, I thought it would be a good project to try to write my own widget.

Behold!

I still have a few bugs to iron out. p5js-player is on github.

Posted in computing, programming | Leave a comment

Embedding p5.js processing sketches in WordPress posts

Posted on January 7, 2017 by Nick

Yesterday I discovered the p5.js project by watching this video series. I got excited and wrote my first sketch “bouncing points,” a classic that I once wrote as a java applet back in the 1990’s. Very simple to do with p5.js. I look forward to writing more experiments!

javascript works fine in WordPress, one just switches to text mode while editing the post and paste in <script> tags and it works fine. p5.js will even render the canvas correctly. I did run into a problem though, the p5.js canvas element gets inserted at the bottom left of the page, below all the WordPress content:

As you can see, you might not notice that it’s working unless you scroll all the way down the page!

The solution: object tag

I found a workaround, I uploaded the p5.js code to my website, then in the WordPress post, I use an object tag to embed it into the post:

<object data="p5js/001_bouncing_points/index.html" width="500" height="275">
    ERROR: failed to load html object!
</object>

ERROR: failed to load html object!

</object>

You have to size the object tag large enough for the created canvas to show up without scrollbars but it looks good on my PC.

Look, I can even insert a 2nd canvas!

It also looks to be running fine on my Android phone! Success!

Posted in computing | 10 Comments

Nsound rewrite 2016

Posted on June 25, 2016 by Nick

The Nsound code base is old, (10 years! I can hardly believe that). Obviously I’m a much better developer today (:P), and there’s lots of things that need cleaning. Maintenance is becoming a burden, the build process is complex, it’s difficult to debug.

So I am about to start to rewrite all of nsound, here are some of the directions I want to go:

C++14
float32 only (float64 is overkill, perhaps Buffer<T>?)
header only (avoid the need for .so or .dll)
serialize with JSON
switch to Boost.Python (dump swig)
create generic filter kernel objects
- separate kernel design from filter objects
generic functions on std::vector or std::array
- provide scipy-like signals module suite
combine AudioStream and Buffer into one object, no need to maintain both
Boost unit tests from the beginning
rewrite all the SCons files to keep it simple stupid!
- move scons utilities & tests into proper tools (and share with the rest of the world!)

You can help out too! Working on open source is a great way to strengthen your skills!

Posted in nsound, programming | Leave a comment

Nsound broken on Ubuntu 16

Posted on June 25, 2016 by Nick

I’m sad to report that nsound is broken by default on Ubuntu 16 until I get time to fix it. The main issue is that the default swig package is swig3.0, for some reason this breaks most things in the Python module.

I’ve created an issue here.

Installing the swig2.0 package fixes most of the problems.

Posted in nsound, programming | Leave a comment

Stay GUI neutral with mplapp and Matplotlib

Posted on May 1, 2016 by Nick

I’ve just released a package of widgets implmented with Matplotlib, this allows you to create simple interactive applications without having to write specific GTK, TK, QT code. It provides simple layout with horizontal and vertical boxes, just stuff widgets in them and it will do the layout for you.

The code is on GitHub: mplapp

Install with pip:

pip install --user mplapp

1	pip install --user mplapp

Posted in Uncategorized | Leave a comment

Python pith

Posted on February 28, 2016 by Nick

I just wrote a little tool to help with your Python development called pith.

pip install --user pith

1	pip install --user pith

Checkout the release.

Posted in programming | Leave a comment

Ranking Photos With Python

Posted on February 20, 2016 by Nick

Photo Ranking With Python

What is this?

This is a tool that uses the Elo Ranking System written in Python using:

Matplotlib
Numpy
exifread

Features:

Auto image rotation that the camera recored in the EXIF meta data
Persistent state from execution to execution so you can pickup where you left off
New photos that are added to the photo dir after initial ranking are picked up

Install dependencies

Use your system’s package manager to install Matplotlib & Numpy if you don’t already have them installed.

Next, you can use pip to install the EXIF image reader package exifread:

pip install exifread --user

				1

						pip install exifread --user

How to rank photos

Once you have to dependencies installed, run rank_photos.py on the command line passing it the directory of photos.

./rank_photos.py -h

usage: rank_photos.py [-h] [-r N_ROUNDS] [-f FIGSIZE FIGSIZE] photo_dir

Uses the Elo ranking algorithm to sort your images by rank. The program globs
for .jpg images to present to you in random order, then you select the better
photo. After n-rounds, the results are reported.

positional arguments:
  photo_dir             The photo directory to scan for .jpg images

optional arguments:
  -h, --help            show this help message and exit
  -r N_ROUNDS, --n-rounds N_ROUNDS
                        Specifies the number of rounds to pass through the
                        photo set (3)
  -f FIGSIZE FIGSIZE, --figsize FIGSIZE FIGSIZE
                        Specifies width and height of the Matplotlib figsize
                        (20, 12)

				
					
				1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19

						./rank_photos.py -h
 
usage: rank_photos.py [-h] [-r N_ROUNDS] [-f FIGSIZE FIGSIZE] photo_dir
 
Uses the Elo ranking algorithm to sort your images by rank. The program globs
for .jpg images to present to you in random order, then you select the better
photo. After n-rounds, the results are reported.
 
positional arguments:
  photo_dir             The photo directory to scan for .jpg images
 
optional arguments:
  -h, --help            show this help message and exit
  -r N_ROUNDS, --n-rounds N_ROUNDS
                        Specifies the number of rounds to pass through the
                        photo set (3)
  -f FIGSIZE FIGSIZE, --figsize FIGSIZE FIGSIZE
                        Specifies width and height of the Matplotlib figsize
                        (20, 12)

					

			

For example, iterate over all photos three times:

./rank_photos.py ~/Desktop/example/

				1

						./rank_photos.py ~/Desktop/example/

After the number of rounds complete, ranked.txt is written into the photo dir.

Ranking work is cached

After completing N rounds of ranking, a file called ranking_table.json is written into the photo dir. The next timerank_photos.py is executed with the photo dir, this table is read in and ranking can continue where you left off.

You can also add new photos the the directory and they will get added to the ranked list even though they weren’t included previously.

Example

Suppose there is a dir containing some photos:

ls -1 ~/Desktop/example/

20160102_164732.jpg
20160109_151557.jpg
20160109_151607.jpg
20160109_152318.jpg
20160109_152400.jpg
20160109_152414.jpg
20160109_153443.jpg

				
					
				1
2
3
4
5
6
7
8
9

						ls -1 ~/Desktop/example/
 
20160102_164732.jpg
20160109_151557.jpg
20160109_151607.jpg
20160109_152318.jpg
20160109_152400.jpg
20160109_152414.jpg
20160109_153443.jpg

					

			

These photos haven’t been ranked yet, so lets rank them, in this example, 1 round:

./rank_photos.py -r 1 ~/Desktop/example/

				1

						./rank_photos.py -r 1 ~/Desktop/example/

Once the number of rounds complete, the ranked list is dumped to the console:

Final Ranking:
Rank    Score    Matches    Win %    Filename
   1    1433          2     100.00    20160109_152414.jpg
   2    1414          3      66.67    20160109_151557.jpg
   3    1401          2      50.00    20160109_153443.jpg
   4    1400          2      50.00    20160102_164732.jpg
   5    1387          3      33.33    20160109_151607.jpg
   6    1383          3      33.33    20160109_152318.jpg
   7    1382          3      33.33    20160109_152400.jpg

				
					
				1
2
3
4
5
6
7
8
9

						Final Ranking:
Rank    Score    Matches    Win %    Filename
   1    1433          2     100.00    20160109_152414.jpg
   2    1414          3      66.67    20160109_151557.jpg
   3    1401          2      50.00    20160109_153443.jpg
   4    1400          2      50.00    20160102_164732.jpg
   5    1387          3      33.33    20160109_151607.jpg
   6    1383          3      33.33    20160109_152318.jpg
   7    1382          3      33.33    20160109_152400.jpg

					

			

The ranked list is also written to the file ranked.txt. The raw data is cached to the file ranking_table.json:

cat ~/Desktop/example/ranking_table.json

{
    "photos" : [
        {
            "matches" : 2,
            "wins" : 2,
            "score" : 1432.736306793522,
            "filename" : "20160109_152414.jpg"
        },
        {
            "matches" : 3,
            "wins" : 2,
            "score" : 1413.760501639972,
            "filename" : "20160109_151557.jpg"
        },
        {
            "matches" : 2,
            "wins" : 1,
            "score" : 1400.736306793522,
            "filename" : "20160109_153443.jpg"
        },
        {
            "matches" : 2,
            "wins" : 1,
            "score" : 1400.0336900375303,
            "filename" : "20160102_164732.jpg"
        },
        {
            "matches" : 3,
            "wins" : 1,
            "score" : 1387.00607880615,
            "filename" : "20160109_151607.jpg"
        },
        {
            "matches" : 3,
            "wins" : 1,
            "score" : 1383.263693206478,
            "filename" : "20160109_152318.jpg"
        },
        {
            "matches" : 3,
            "wins" : 1,
            "score" : 1382.4634227228255,
            "filename" : "20160109_152400.jpg"
        }
    ]
}

				
					
				1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48

						cat ~/Desktop/example/ranking_table.json
 
{
    "photos" : [
        {
            "matches" : 2,
            "wins" : 2,
            "score" : 1432.736306793522,
            "filename" : "20160109_152414.jpg"
        },
        {
            "matches" : 3,
            "wins" : 2,
            "score" : 1413.760501639972,
            "filename" : "20160109_151557.jpg"
        },
        {
            "matches" : 2,
            "wins" : 1,
            "score" : 1400.736306793522,
            "filename" : "20160109_153443.jpg"
        },
        {
            "matches" : 2,
            "wins" : 1,
            "score" : 1400.0336900375303,
            "filename" : "20160102_164732.jpg"
        },
        {
            "matches" : 3,
            "wins" : 1,
            "score" : 1387.00607880615,
            "filename" : "20160109_151607.jpg"
        },
        {
            "matches" : 3,
            "wins" : 1,
            "score" : 1383.263693206478,
            "filename" : "20160109_152318.jpg"
        },
        {
            "matches" : 3,
            "wins" : 1,
            "score" : 1382.4634227228255,
            "filename" : "20160109_152400.jpg"
        }
    ]
}

					

			

If you run the program again, the cached data is loaded and new match ups can be continued using the cached data. If new photos were added, they also get added to the table data and are included in match ups.

The code has been posted to github.

Posted in computing, programming | Leave a comment

Listening to Gravity Waves with Nsound!

Posted on February 12, 2016 by Nick

Gravity Wave Signal Processing

With all the excitement about LIGO and gravity waves, I’ve downloaded the raw GW150914 detection data for from here. The file I downloaded was 32 seconds sampled at 4096 samples per second.

Next I reviewed some of the LIGO tutorial on signal processing here, and proceeded to write a Python script using Nsound to process the data. Talk about finding a needle in a haystack!

Raw time-series data plot, the detection event occurs around 16.5 seconds in this plot:

Where’s the signal? It’s buried in noise! Mostly low frequencies. But with the help of a band pass filter from 80 to 280 Hz, we can get a hint of the signal:

Zooming in:

Plotting the spectrogram:

Audio

And finally, we can listen to the universe! The detection is a single blip as show above, however for your listening pleasure I’ve repeated the blip 4 times:

Link to the .wav here.

Python Code

Here is the Python code I used which uses my Nsound Python library:

"""
Reads in the LIGO gravity wave data for GW150914, a file named
'H-H1_LOSC_4_V1-1126259446-32.txt', and performs a band pass filter from 80 to
280 Hz. The data are then written out to a .wav file so it can be played on a
computer.

By Nick Hilton
2016-02-11
"""

import Nsound as ns
plt = ns.Plotter()

#------------------------------------------------------------------------------
# read in time series data

with open('H-H1_LOSC_4_V1-1126259446-32.txt', 'r') as fd:
    lines = fd.readlines()

sr = 4096.0

a = ns.AudioStream(sr, 1)

for l in lines:
    l = l.strip()

    # skip comments
    if l.startswith('#'):
        continue

    s = float(l)

    a << s

a.normalize()

# The data don't start and end at zero so there's an initial step function, which
# causes a big spike in the spectrogram.  So I'm going to apply a Gaussian
# window to smooth the start and stop of the data.

dur = a.getDuration()

gen = ns.Generator(sr)

gau = gen.drawFatGaussian(dur, 0.99)

while len(gau) < len(a):
    gau << 0.0

a *= gau

#-----------------------------------------------------------------------------
# band pass filter to region of interest

bpf = ns.FilterBandPassIIR(sr, 5, 80, 280, 0.01)

a2 = bpf.filter(a)

a2.normalize()

a2.plot('Filtered Time Series GW150914')

plt.show()

#-----------------------------------------------------------------------------
# spectrogram

spec = ns.Spectrogram(a2[0], sr, 0.0625, 0.0039, ns.PARZEN)
spec.plot('Gravity Freakin\' Waves!', False, 0.5)

# zoom in on region
plt.xlim(16.15, 16.63)
plt.ylim(15, 460)

a2.plot('Time Domain Signal')

# zoom in on region
plt.xlim(16.15, 16.63)

#------------------------------------------------------------------------------
# pull out 2 second around the gravity wave and repeat it several times for
# playback

a3 = a2.substream(16.0, 2.0)

# remove any step function around the edges

gau = gen.drawFatGaussian(2.0, 0.99)

while len(gau) < len(a3):
    gau << 0.0

a3 *= gau

a3 << a3 << a3

a3 >> "out.wav"

ns.Plotter.show()

"""

Reads in the LIGO gravity wave data for GW150914, a file named

'H-H1_LOSC_4_V1-1126259446-32.txt', and performs a band pass filter from 80 to

280 Hz. The data are then written out to a .wav file so it can be played on a

computer.

By Nick Hilton

2016-02-11

"""

import Nsound as ns

plt = ns.Plotter()

#------------------------------------------------------------------------------

# read in time series data

with open('H-H1_LOSC_4_V1-1126259446-32.txt', 'r') as fd:

lines = fd.readlines()

sr = 4096.0

a = ns.AudioStream(sr, 1)

for l in lines:

l = l.strip()

# skip comments

if l.startswith('#'):

continue

s = float(l)

a << s

a.normalize()

# The data don't start and end at zero so there's an initial step function, which

# causes a big spike in the spectrogram. So I'm going to apply a Gaussian

# window to smooth the start and stop of the data.

dur = a.getDuration()

gen = ns.Generator(sr)

gau = gen.drawFatGaussian(dur, 0.99)

while len(gau) < len(a):

gau << 0.0

a *= gau

#-----------------------------------------------------------------------------

# band pass filter to region of interest

bpf = ns.FilterBandPassIIR(sr, 5, 80, 280, 0.01)

a2 = bpf.filter(a)

a2.normalize()

a2.plot('Filtered Time Series GW150914')

plt.show()

#-----------------------------------------------------------------------------

# spectrogram

spec = ns.Spectrogram(a2[0], sr, 0.0625, 0.0039, ns.PARZEN)

spec.plot('Gravity Freakin\' Waves!', False, 0.5)

# zoom in on region

plt.xlim(16.15, 16.63)

plt.ylim(15, 460)

a2.plot('Time Domain Signal')

# zoom in on region

plt.xlim(16.15, 16.63)

#------------------------------------------------------------------------------

# pull out 2 second around the gravity wave and repeat it several times for

# playback

a3 = a2.substream(16.0, 2.0)

# remove any step function around the edges

gau = gen.drawFatGaussian(2.0, 0.99)

while len(gau) < len(a3):

gau << 0.0

a3 *= gau

a3 << a3 << a3

a3 >> "out.wav"

ns.Plotter.show()

Posted in computing, nsound, programming, science | Leave a comment

weegreenblobbie.com

Swig is broken in Ubuntu 16.04 (xenial)

Building jEdit from scratch

Getting the jEdit code

Building jEdit

Upgrading Apache Ant on Ubuntu 16.04

P5js player widget in pure javascript

Embedding p5.js processing sketches in WordPress posts

The solution: object tag

Nsound rewrite 2016

Nsound broken on Ubuntu 16

Stay GUI neutral with mplapp and Matplotlib

Python pith

Ranking Photos With Python

Photo Ranking With Python

What is this?

Install dependencies

How to rank photos

Ranking work is cached

Example

Listening to Gravity Waves with Nsound!

Gravity Wave Signal Processing

Audio

Python Code

Archives

Meta