Category Archives: ubuntu / debian / linux

Installing Ubuntu on a Dell M3800

For a while I was having problems with my laptop having too little memory. It was a Dell XPS13 developer edition. A system I was very happy with, but the 8GB RAM was proving to be too little in some cases. So I wanted something more powerful. I chose the Dell M3800 because it has a recent i7, has 16GB RAM and a HiDpi screen with a 3200×1800 resolution. No “developer edition” though but (being a full-time Ubuntu Linux user) I went bought it anyway.

I installed Ubuntu 14.10 on it. and encountered a few issues.

  • The system was a bit slower to compile than my previous one even though it has more memory and a faster processor. Looking into this, it seems this was caused by overly aggressive settings to keep the temperature down. I tweaked the termald settings to kick in at 75 instead of 45 degrees and this halved the compile time! For details look at “Thermal Issues” and “Powerclamp”.
  • Parcellite (a tool which keeps clipboard memory) does not seem to work. I replaced this with diodon which seems to be a better tool anyway.
  • The system needs to be rebooted a lot more than my previous one. There are two problems which sometimes occur which need a reboot to fix. One is that the screen turns black – as if I am not doing anything except it kicks in after a second or two instead of five minutes. I turns back on when I move the mouse but gets annoying quickly. The other is that the wifi sometimes gets very disfunctinal, hardly being able to connect to the local network and when it does only has very limited bandwidth.
  • Sometimes the keyboard becomes quite unresponsive. This may be application related. I have encountered this most in pgAdmin and Inkscape. Inkscape also seems to crash very easily.

The HiDpi screen is a challenge in itself.

  • Ubuntu itself (or Unity) looks great. The trick is to set the “Scale for menu and title bars” setting in the display menu. I used “2” as setting so the screen looks as if I have a resolution of 1600×900 but with more detailed rendering.

  • Firefox also works fine. You just have to set “layout.css.devPixelsPerPx” to 2 in about:config.
  • The same trick works for thunderbird. To access about:config, go to Edit → Preferences → Advanced → Config editor.
  • Chromium only half works. Many windows and pop-up boxes are positioned and scaled wrong.
  • Unfortunately many application don’t (or only half) support the scale setting. For example pgAdminIII is mostly ok, but all rows in query results are only half the size of the text displayed in them.
  • Java applications also don’t respect the scale. In some cases this can be fixed by configuring the text size (for example in IntelliJ IDEA), but the menus and icons stay too small. For the many, installing jayatana (Unity global menu integration) really helps.

In short, the linux support for HiDpi displays is clearly a work in progress.

Migrating from Oracle to Postgres for a Java Hibernate application

For an old project which is Spring/Hibernate based, there was a need to migrate from an Oracle database to a PostgreSQL database.

The initial task was to migrate the database. I used ora2pg for this. This is a command line tool which allows you to extract your Oracle database to SQL. It was somewhat difficult to install on my Ubuntu system (because I needed the Perl Oracle client – once I got that fully installed it was a breeze, these two articles helped me a lot).

Ora2pg uses a configuration file. You need to fill in the correct for ORACLE_DSN, ORACLE_USER and ORACLE_PWD. Do not forget to also set the SCHEMA value (likely to the same value as ORACLE_USER).
To control the data which needs to be exported, you have to set the TYPE.
I first exported the table structure using “TABLE”. I then copied the result file (to prevent it from being overwritten) and then ran ora2pg again using “INSERT” as type to export the data.

Before being able to use these files, I needed to convert them to UTF-8. This was done using the command

recode ISO-8859-1..UTF8 *.sql

To create the database in Postgres I started by splitting the file with the table structure in a file which only creates the table structure itself and one which contains the constraints. I also had to extract the tail of output file with the data which contains the new values for the sequences. It was then possible to rebuild the database by first creating the structure, then inserting the data and lastly rebuilding the constraints.

psql wd_test -f output-table-structure-structure.sql
psql wd_test -f output.sql
psql wd_test -f output-table-structure-constraints.sql

To finish off I still had to add the necessary sequences (you should be able to get them though ora2pg again though I used the Hibernate generated DDL) and reset the sequences to the correct values.

Last step of the data migration was to compare the Hibernate generated DDL so see whether all constraints exist. I particularly has to add a primary key in all tables, foreign key constraints and define various unique constraints.

Then test your application.

I encountered two problems. First, there are some functions which were not available. I checked all HQL and native queries to search for functions which are used. There were two missing for this application, month and year. These had to be defined.

CREATE OR REPLACE FUNCTION MONTH(TIMESTAMP without TIME ZONE) RETURNS INTEGER
AS $$
      SELECT EXTRACT(MONTH FROM $1)::INTEGER;
$$ LANGUAGE SQL IMMUTABLE;
 
CREATE OR REPLACE FUNCTION MONTH(TIMESTAMP WITH TIME ZONE) RETURNS INTEGER
AS $$
      SELECT EXTRACT(MONTH FROM $1)::INTEGER;
$$ LANGUAGE SQL STABLE;
 
CREATE OR REPLACE FUNCTION MONTH(DATE) RETURNS INTEGER
AS $$
      SELECT EXTRACT(MONTH FROM $1)::INTEGER;
$$ LANGUAGE SQL IMMUTABLE;
 
 
CREATE OR REPLACE FUNCTION YEAR(TIMESTAMP without TIME ZONE) RETURNS INTEGER
AS $$
      SELECT EXTRACT(YEAR FROM $1)::INTEGER;
$$ LANGUAGE SQL IMMUTABLE;
 
CREATE OR REPLACE FUNCTION YEAR(TIMESTAMP WITH TIME ZONE) RETURNS INTEGER
AS $$
      SELECT EXTRACT(YEAR FROM $1)::INTEGER;
$$ LANGUAGE SQL STABLE;
 
CREATE OR REPLACE FUNCTION YEAR(DATE) RETURNS INTEGER
AS $$
      SELECT EXTRACT(YEAR FROM $1)::INTEGER;
$$ LANGUAGE SQL IMMUTABLE;

Lastly, there were count queries which had an order defined on them. Typically the query was built using generic code and either executed normally or using a count projection. This works perfectly in Oracle but Postgres complains saying

column “tableName.fieldName” must appear in the GROUP BY clause or be used in an aggregate function

Fixing the code for this seemed non-trivial and possible cause of bugs, so I introduced “patch” code which removes the order from the query and is called before applying the count projection.

public final class PgCompatibility {
 
    private static final Logger LOG = LoggerFactory.getLogger(PgCompatibility.class);
 
    private PgCompatibility() {
        // hide constructor
    }
 
    /**
     * Criteria from which the order needs to be removed.
     * <p/>
     * On queries like "select count(*) from TD_MAANDPRESTATIE order by D_EERSTE_KALENDERDAG asc" the order is
     * irrelevant. Oracle seems to recognize this while Postgres gives an error like
     * "column "d_eerste_kalenderdag" must appear in the GROUP BY clause or be used in an aggregate function"
     *
     * @param criteria criteria from which the order needs to be removed if possible
     */
    public static void removeOrder(Criteria criteria) {
        if (criteria instanceof CriteriaImpl) {
            try {
                Field orderEntries = CriteriaImpl.class.getDeclaredField("orderEntries");
                makeAccessible(orderEntries);
                ((List) orderEntries.get(criteria)).clear();
            } catch (NoSuchFieldException nsfe) {
                LOG.error("Cannot find field.", nsfe);
            } catch (IllegalAccessException iae) {
                LOG.error("Cannot access field.", iae);
            }
        }
    }
 
    /**
     * Make the given field accessible, explicitly setting it accessible if necessary.
     * The setAccessible(true) method is only called when actually necessary, to avoid unnecessary
     * conflicts with a JVM SecurityManager (if active).
     * <p/>
     * This method is borrowed from Spring's ReflectionUtil class.
     *
     * @param field the field to make accessible
     * @see java.lang.reflect.Field#setAccessible
     */
    public static void makeAccessible(Field field) {
        if ((!Modifier.isPublic(field.getModifiers()) || !Modifier.isPublic(field.getDeclaringClass().getModifiers()) ||
                Modifier.isFinal(field.getModifiers())) && !field.isAccessible()) {
            field.setAccessible(true);
        }
    }
}

Presto, migration worked.

Notification when maven build finishes

Maven builds often take some time. This often causes me to start doing something else (read mail, browse web), which makes me unaware of when the build completes and I can continue working.

There should be a way to fix this. On Ubuntu, you get notifications when mail arrives, when network is up/down etc. Why not use this same system to get notifications when the maven build completes.

Maven notification window

Maven notification window

You can do this by adding the following in your ~/.bash_aliases file:

alias maven="command mvn"
notified_maven() {
  maven $* ; notify-send --icon=message-im "mvn" "build finished"    
}
alias mvn=notified_maven

Update
Thanks to Adrien, you can also use a more advanced version, showing the build result and maven parameters.

alias maven="command mvn"
notified_maven() {
  maven $* | \
  perl -pe'$m|=/BUILD .*SUCCESS/; END {exit!$m}' && \
  notify-send --icon=face-cool "`basename $(pwd)`: mvn $*" "Build SUCCESS" || \
  notify-send --icon=face-crying "`basename $(pwd)`: mvn $*" "Build FAILED"
}
alias mvn=notified_maven