Thursday, September 17, 2015

double encoding of UTF8 strings

Turns out I was focusing on the wrong leads. These strings became values in a hash which was run through JSON->encode before saving to db. What prompted the re-encoding was not the values of the hash, but the keys. These were string literals in my program which were not marked as UTF8. Perl looked at my keys, looked at my values, saw a mixed bag, and decided to run the whole thing through the shredder again, just to be on the safe side.

The solution was simple:
use utf8

Because my keys were source-code string literals, I want my source code to be UTF8. use utf8 assures that.

thanks to:

Monday, August 10, 2015

how to debug anonymous functions and coderefs in Perl 5

use strict;
use warnings;
use B::Deparse;

sub debug_coderef {
    my $possibly_code = shift; 
    my $deparse = B::Deparse->new();
    if (ref $possibly_code && ref $possibly_code eq 'CODE') {
        print STDERR "CODEREF: ", $deparse->coderef2text($possibly_code), "\n";
    }  else {
        print STDERR "NOT A CODEREF: ", Dumper($possibly_code), "\n";

debug_coderef(sub {print "a";});

Wednesday, June 3, 2015

perl XS adventures, episode 1

Perl XS is easy if you read the docs in the proper order. What is the proper order ? I have no idea, I kept at it until it clicked. Well, not all of it clicked, just enough to get some results.

Summary so far: managed to persuade perl to run my code from the .xs file, to compile my .c files and to call code in the .c files from the .xs file.

To have it work I kept it simple. I tried the not-simple version and you don't want to know the story.

Hints for the inpatient

Quick hints

  • man 2 and man 3 are your best friends, if you don't know this already; for example
    man 2 readdir
    man 3 strcat
  • read perlguts before perlapi; better, do not attempt to read perlapi end-to-end, just consult it when you need
  • normal C code in custom .c and .h files, which all got to the root of the project (where Makefile.PL is); it is possible to put them some other place but I have not found out how to change Makefile.PL to deal with it.
  • include .h files in the .xs file
  • the code that uses Perl types and functions stays in the .xs file; Perl-independent C code can go before the
    MODULE = My::Module  PACKAGE = My::Module
  • extra libraries added in the Makefile as obiect files in here
    OBJECT            => '$(BASEEXT)$(OBJ_EXT) blah.o', 
    or even better
    OBJECT            => '$(BASEEXT)$(OBJ_EXT) blah$(OBJ_EXT)'
  • to raise exceptions use
    croak(const char * blah)
    instead of exit(1) in the code in the .xs files

How to change the generated Makefile

based on

Add a new compilation target

sub postamble {
    # this should be empty 
    my $inherited = shift->SUPER::postamble(@_);

    my $extra = q{
clean_temp_files :
 if [ -d /tmp/blah ]; then rm -rf /tmp/blah; fi
    return $inherited . "\n" . $extra;

Modify an existing compilation target

I think all the sections in the generated Makefile can be intercepted and modified, but so far I have tried only with clean:

sub clean {
  my $inherited = shift->SUPER::clean(@_);
  $inherited =~ s/clean :: clean_subdirs/clean :: clean_subdirs clean_temp_files/;
  return $inherited;
Step by step instructions for the very patient XS Mechanics or XS Fun.

Monday, May 18, 2015

searching all skype messages on Linux

if you ever need to search skype messages:


the query is "SELECT author, datetime(timestamp, 'unixepoch'), body_xml FROM messages ORDER BY timestamp desc;"

the sqlite file is ~/.Skype/your_user_name/main.db

Might need to stop skype before looking at the file.

Monday, February 23, 2015

how to install DLangIDE on Debian Wheezy

how to install DLangIDE on Debian Wheezy:

 - get the source from here: or

git clone
cd dlangide
dub run

This will fail with

Derelict: cannot load freetype shared library: Failed to load symbol FT_Gzip_Uncompress from shared library


2015-02-23 16:16:06.462 E No font files found!!!
2015-02-23 16:16:06.463 E Currently, only hardcoded font paths implemented.
2015-02-23 16:16:06.463 E Probably you can modify sdlapp.d to add some fonts for your system.


Failed to load symbol SDL_HasAVX from shared library


SDL_CreateWindow failed - cannot create OpenGL window: No OpenGL support in video driver

No worries, will fix them next.

 - set the LD_LIBRARY_PATH to include /usr/local/lib if it does not include it already

export LD_LIBRARY_PATH=/usr/local/lib/:$LD_LIBRARY_PATH

 - install freetype from sources (configure, make, sudo make install) from here

 - install libxext-dev (required by SDL)

apt-get install libxext-dev

 - install libsdl2-dev from wheezy-backports: don't need it since we'll install from sources, but lots of other dependencies will be installed

apt-get -t wheezy-backports install libsdl2-dev

 - install SDL2 from sources (configure, make, sudo make install) from here

 - install ttf-dejavu ttf-dejavu-core ttf-dejavu-extra

apt-get install ttf-dejavu ttf-dejavu-core ttf-dejavu-extra

 - update dependencies

dub upgrade --force-remove

 - edit a file in DLangUI:

find the source for DLangUI in ~/.dub/packages/ and edit the file src/dlangui/platforms/sdl/sdlapp.d :

find the line "ft.registerFonts("/usr/share/fonts/dejavu/");" and add after it this line:


return to the folder with the sources for DLangIDE and run

dub build --force

and then

dub run



and you should get the main window to open

Did not get further today, will update this post if I get into more snags.

To get autocompletion install dcd from .

Tuesday, February 10, 2015

format json with Gedit and Perl

How to prettify/format/indent json with Gedit and Perl:

install JSON::XS
go to Plugins and enable "External tool"

go to Tools -> Manage External Tools -> add new (the button in the bottom on the left if no icons are shown)
add a name, for example "Format JSON with Perl"
replace the contents of the text area on the right with

#!/usr/bin/env perl 

use strict;
use JSON::XS;
local $/ = undef;
my $content = <STDIN>;

my $j = JSON::XS->new->utf8->pretty(1);
my $output = $j->encode($j->decode($content));

print $output;

open your file, then Tools -> External Tools -> "Format JSON with Perl"

you can also set a shortcut in the Manage External Tools window

Thursday, January 15, 2015

my perlbrew build options

mostly copied from Debian, as shown by

perl -V

here they are:

perlbrew install -j 5 -Dusethreads -Duselargefiles -Dcccdlflags=-fPIC -Duse64bitint -Dpager=/usr/bin/sensible-pager -Uafs -Ud_csh -Ud_ualarm -Uusesfio -Uusenm -Doptimize=-O2 -Duseshrplib perl-5.20.1

"-j 5" thanks to

fglrx (AMD proprietary driver) on Debian Wheezy on Athlon 5350 with AM1H-ITX

The open source radeon driver works fine, but if for some reason you want to install the proprietary driver the Wheezy default will not recognize the integrated graphics AMD Radeon HD 8400. Fortunately the version in wheezy-backports does.

Add this line to /etc/apt/sources.list:

deb wheezy-backports main contrib non-free


apt-get update

apt-get install -t wheezy-backports fglrx-driver fglrx-control

aticonfig --initial


Yes, the reboot is required, the open source radeon driver needs to be blacklisted and the package fglrx-driver apparently does that for you.

Friday, January 9, 2015

convert .msg files from Outlook to a format readable on linux with a text editor

How to convert .msg files from Outlook to a format readable on linux with a text editor:


cpanm Email::Outlook::Message


cpan -i Email::Outlook::Message

or on Debian and derivatives such as Ubuntu:

apt-get install libemail-outlook-message-perl


msgconv my_unreadable_binary_file.msg
less my_unreadable_binary_file.eml