trying to solve IT problems

How I tried to fix certain programming problems, mostly in the java, JEE, JBoss scene, web area and using Ubuntu or Debian linux.

  • Home
  • Geomajas / GIS
  • About
Twitter RSS
Category Archives: java

December BeJUG notes: Let’s make this test suite run faster

Posted on February 22, 2012 by joachim
No Comments

Let’s make this test suite run faster, David Gageot, @dgageot

Continuous testing: infinitest
run test automatically in the IDE when you make changes

Cheaters:

build up to 4 modules in parallel
mvn -T4 clean install
but careful with tests with side effects!

surefire plugin : run tests in parallel

classes 2
but careful with tests with side effects!

Lazy:

delete useless tests

delete dead code (and remove the tests for that code)
use statistics to figure out which features are not used

work in a sandbox
- database is slow (e.g. use in memory H2 which is very similar MySQL)
- NoSQL: choose a server that runs in memory, eg Voldemort
- SMTP server: eg SubEtha SMTP
- file system: eg Apache VFS, Spring resource
do everything in memory, as a bonus it is easier to parallelize tests

Brave:

5 minutes a day can make a difference

don’t test business rules in integration tests – do it in unit tests

Break longer integration test into a faster integration test and small unit tests

Mock the slowest layers, e.g. with Spring and Mockito spies
- in post processor, wrap beans with a spy

Don’t test through the browser, Selenium is often overkill

Action #0 Simplify and optimize your code

Part II : preferred testing techniques
————————————–

MoreUnit eclipse plugin to switch between class and test
CTRL-Shift-T in Intellij to move to test

infinitest (both eclipse and IntelliJ)
automatically run tests which are impacted by a code change

junit max
like infinitest but runs all tests, sorts tests (failing most and faster tests first)

hamcrest to build more readable assertions
assertThat(“hello world”, is(equalsTo(Greeter.greating)));

FEST-assert to build even more readable assertions (better than hamcrest)
assertThat(yoda).isInstanceOf(Jedi.class)
.isEqualTo(foundJedi);

http://fest.easytesting.org/

JUnit 4.10
@Theory, @DataPoint (nightmare)
@Rule : applied to all tests in class, can be changed during the test (in code)
@SuiteClasses
@Category (beta)

Categories: java

Geomajas GIS framework 2011 roundup

Posted on December 30, 2011 by joachim
No Comments

2011 was a very productive year for the Geomajas project. There was a lot of progress in many areas.

Before listing everything that happened in the last year, let’s start on what did not happen. There was no new release in the last six months. That is not good. There have been some great developments, just no releases. As a community we should try to work more towards releases. On the up side, similar to the bout of releases we saw last June, the same should happen in January 2012.

Summary

  • 2 back-end releases.
  • 2 GWT face releases.
  • 1 face saw it’s initial release.
  • 1 face newly added but not yet released.
  • many plug-in releases.
  • 2 plug-in saw it’s initial release.
  • 8 plug-ins added but not yet released.
  • 3 separate Open Source projects started (but not yet released) which are usable without the Geomajas back-end.

Project structure

The Geomajas showcase application is being reworked to match the project structure better. Instead of being a monolithic application, it is now changed into an aggregator. There is now a base for building examples. When this is used by the individual plug-ins (packaged in the correct way), then including the latest examples in the showcase can be done by changing a dependency. This should allow us to release the showcase more often and still keep it more stable, doing the testing for the panels in the plug-in which is demonstrated.

There is also an ongoing effort to split out code which may be usable in other projects into separate projects. Currently there are three such projects already, a API annotations project, a SLD project and a geometry object project. This way we want to encourage adoption and code sharing. The exact license to be used for these projects is still being discussed.

Back-end

There were two back-end releases in 2011. Version 1.8.0 was the result of the the work done in 2010. It mainly included various smaller improvements which increase the capabilities of the plug-ins and bug fixes. In June version 1.9.0 was released. This release has a lot of performance improvements, mixed geometry layers and some usability improvements.

In trunk there are many changes including the support for SLD (using a separate SLD project which handles
reading and writing SLD objects and stores them in DTOs). The API annotations and Geometry DTOs have also been moved to separate projects. There is now a service and command to get legend images and a resource controller to make it more predictable how to retrieve resources from both the client and server side. There are various improvements for usability and testing. There are also many bug fixes.

Faces

GWT face

Like the back-end, the GWT face also saw two releases and a lot of not-yet-released changes.

The 1.8.0 release contained a lot of bug fixes, the ability to have multiple passive controllers (or listeners) and an update to the then latest GWT version.

1.9.0 included some improvements which are important for rasterizing and to improve caching. It also has some additions to make feature form building a lot more flexible (both in layout and to allow configuring the widget to use for individual fields). An example was also created to demonstrate the integration with other web frameworks.

In trunk various improvements have been added for both developer and end-user easy of use. We have tracked the releases of GWT and SmartGWT (2.4 and 2.5). The map rendering has been reworked to make it more efficient. Many parts have been split out. General widgets have been moved to the new utility widgets plug-in and some general GWT (and not SmartGWT) stuff has been moved to the common GWT module to improve code sharing with the new PureGWT face. The exception window and log utility have been made more practical. The command dispatcher has been updated to handle retries when the authentication token times out. For improved tablet support, the controllers now handle both mouse and touch events. All style related code has been extracted to allow customization.

dojo face

Considering that this face had not seen any significant attention since the 1.6.0 release of the back-end, and that there was no module maintainer for even longer), it was decided by the community to move the module to the attic.

PureGWT face

This is a wonderful new effort to build a face which is more suitable for mobile use. Contrary to the GWT face it does not use a widget library. This gives a big bonus in download size for the page and has some extra advantages like allowing the full use of all GWT load time enhancements.

A very early version was shown by Pieter at FOSS4G in Denver. It was only tested on Android but also proved to work just fine on the iPad during the talk. It was also used in the routing demo at FOSS4G.

There is a lot of work yet to do, but expect a milestone release (working but not fixing the API yet) in January or February.

REST face

The REST face saw it’s initial release. This allows Geomajas layers to be read in GeoJSON. This can for example be used to display the layer in OpenLayers. This was the result of work started at the FOSS4G Barcelona (2010) conference code sprint.

Plug-ins

Most existing plug-in has at least one release in 2011 adding some minor features and bug fixes. Some plug-ins had more notable changes.

GeoTools layer

Version 1.8.0 includes improvements to assure attempts to reconnect to the data store on failure.

In trunk the recovery of data store failures is further improved (some cases still caused problems). This will probably be released in January.

WMS layer

The 1.8.0 release added new features to access secured WMS servers. This also makes sure the credentials are not leaked to the client. Support for GetFeatureInfo was also added.

In trunk the ability is added for WMS tile requests to be cached. This will probably be released in January.

Hibernate layer

The 1.8.0 and 1.8.1 versions were released. They contain performance improvements and bug fixes.

staticsecurity

There has been a release with some changes to make the security context cachable.

In trunk the plug-in has seen major improvements. For starters full documentation has been added. The authentication is now pluggable (allowing users to be added and passwords to be changed dynamically). A module has been added to use LDAP for authentication.

For integration with the GWT face, the login window and logout buttons have been redone to integrate with the mechanisms for storing the current token, getting information about the current user and doing automatic request to login again when the token expires.

An example has been added to show how you can add application specific policies.

New plug-ins

KML layer

The first (known to us) plug-in which is external to the project itself. It is available on github.

caching

A new plug-in was released which allows caching of various things, by default tile and bounds requests. It is fully configurable and pluggable, but by default it uses the Infinispan distributed cache. Caching can be tuned by cache category and layer.

The caching is fully integrated in the system. It knows when data needs to be invalidated and can thus limit the amount of data to invalidate.

rasterizing

This module has been in development throughout most of the year. It can rasterize tiles to render them at the server instead of the client side. This allows more powerful rendering methods than can be done in the browser and (though tight integration with caching) allows performance improvements, putting the calculations close to the data. It is fully integrated with the new SLD support.

We expect an initial release in January or February.

editing

This is a new plug-in which makes the editing of feature much more powerful and more intuitive than it currently is. We expect at least a milestone release January/February.

utility widgets

This contains various not-really-map-related widgets which may be useful in Geomajas applications. Current widgets include CardLayout, a wizard, a ribbon (which is a much nicer and more powerful version of the action bar) etc. We expect at least a milestone release January/February.

reporting

This is a plug-in which integrates JasperResports and Geomajas. You can build your reports using a tool like iReports and integrate maps in the report.
There are not definite release plans (in is not sufficiently complete, lacks documentation and the API is not solid enough), though a milestone release would be possible.

feature info widget

This plug-in sports an advanced feature list table and a nice controller to shows feature information at mouse over.Have a look at the example application in the widget.
There are no definite release plans as some more integration with the back-end and GWT face are needed, the API is not marked and documentation needs to be improved, though a milestone release would be possible.

advanced search and filter widget

This plug-in contains configurable widgets which allow searching and filtering in a more advanced way than the normal stuff included in the back-end and face. For example it gives very easy and user-friendly access to geometric searched based on existing features or geometries which are drawn by the user.
There are no definite release plans as some more integration with the back-end and GWT face are needed, the API is not marked and documentation needs to be improved, though a milestone release would be possible.

advanced view widget

This plug-in contains new widgets for the GWT face that provide a more advanced view of the map. On one hand there is the LegendTree widget, which is a combination of the original layer tree and legend, on the other hand there is a widget that adds support for thematic views.
There are no definite release plans as some more integration with the back-end and GWT face are needed, the API is not marked and documentation needs to be improved, though a milestone release would be possible.

JavaScript API

This plug-in publishes a stable JavaScript API for a Geomajas map widget. This allows easy integration of with other web frameworks easing integration of Geomajas based maps in existing web applications. We expect at least a milestone release January/February.

Community

Though our community is still quite small, we have seen a couple of extra committers this year. We have also seen quite a few new people show up on the forum and the majas-dev mailing list.

For some statistics see the Geomajas page on Ohloh. This shows a the growth or or code base (with a dent when we retired the unmaintained dojo face to the attic). You can also have a look at our mailing list statistics on markmail.

This promises a lot of goodness for 2012.

Happy new year!

Categories: geomajas / GIS, java

Using UnboundID to authenticate against an LDAP server

Posted on December 7, 2011 by joachim
1 Comment

UnboundID is an open source project which provides an SDK for connecting to an LDAP server. Compared with other solutions, UnboundID has two advantages. For starters UnboundId includes an in-memory server which is very practical for unit testing. Secondly (and useful in my practical case) is that the UnboundId API makes it easy to deal with the complexities of an SSL connection.

My use case was to authenticate using LDAP for Geomajas. The easiest solution there is to provide an implementation of the AuthenticationService in the Geomajas staticsecurity plug-in.

Let’s start with the interesting bits, setting up a in-memory server for testing. The server needs to be created. We use a null schema to assure all attributes are allowed. Bind credentials are added for the user we add and a base data is added to the server.

protected InMemoryDirectoryServer server;

@Before
public void before() throws Exception {
  InMemoryDirectoryServerConfig config = new InMemoryDirectoryServerConfig("dc=org");
  config.addAdditionalBindCredentials("cn=test,dc=staticsecurity,dc=geomajas,dc=org", "cred");
  InMemoryListenerConfig listenerConfig = new InMemoryListenerConfig("test", null, PORT, null, null, null);
  config.setListenerConfigs(listenerConfig);
  config.setSchema(null); // do not check (attribute) schema
  server = new InMemoryDirectoryServer(config);
  server.startListening();

  server.add("dn: dc=org", "objectClass: top", "objectClass: domain", "dc: org");
  server.add("dn: dc=geomajas,dc=org", "objectClass: top", "objectClass: domain", "dc: geomajas");
  server.add("dn: dc=roles,dc=geomajas,dc=org", "objectClass: top", "objectClass: domain", "dc: roles");
  server.add("dn: dc=staticsecurity,dc=geomajas,dc=org", "objectClass: top", "objectClass: domain",
      "dc: staticsecurity");
  server.add("dn: cn=testgroup,dc=roles,dc=geomajas,dc=org", "objectClass: groupOfUniqueNames",
      "cn: testgroup");
  server.add("dn: cn=test,dc=staticsecurity,dc=geomajas,dc=org", "objectClass: person", "locale: nl_BE",
      "sn: Tester", "givenName: Joe", "cn: test", "memberOf: cn=testgroup,dc=roles,dc=geomajas,dc=org");
}

@After
public void shutdown() {
  server.shutDown(true);
}

In true TDD style, let’s look at the test for our authentication service before we look at the actual code. The main test (excluding initialization) look like this. For the fluent asserts, the FEST-assert library is used.

@Test
public void testLdapAuthenticationService() throws Exception {
  String password = "bladibla";
  assertThat(service.convertPassword("me", password)).isEqualTo(password);

  assertThat(service.isAuthenticated("wrong", "wrong")).isNull();

  String userId = "test";
  UserInfo authResult = service.isAuthenticated(userId, "cred");
  assertThat(authResult).isNotNull();
  assertThat(authResult.getUserId()).isEqualTo(userId);
  assertThat(authResult.getUserName()).isEqualTo("Joe Tester");
  assertThat(authResult.getUserLocale()).isEqualTo(new Locale("nl_BE"));
  assertThat(authResult.getUserOrganization()).isEqualTo("test");
  assertThat(authResult.getUserDivision()).isEqualTo("person");
  List<AuthorizationInfo> auths = authResult.getAuthorizations();
  assertThat(auths).hasSize(1);
}

On to the actual authentication. On a login request, we connect with the LDAP server and send a bind request to see whether the credentials are valid.

There is a special handling for the case when SSL certificate checks should be ignored. This can be particularly useful for testing. It is not recommended in production as this reduces security (and makes you vulnerable to man-in-the-middle attacks and and attacks using DNS, redirecting to a different server).

public UserInfo isAuthenticated(String user, String password) {
  String userDn = userDnTemplate.replace("{}", user);
  LDAPConnection connection = null;
  try {
    if (allowAllSocketFactory) {
      SSLUtil sslUtil = new SSLUtil(new TrustAllTrustManager());
      connection = new LDAPConnection(sslUtil.createSSLSocketFactory(), serverHost, serverPort);
    } else {
      connection = new LDAPConnection(serverHost, serverPort);
    }

    BindResult auth = connection.bind(userDn, password);
    if (auth.getResultCode().isConnectionUsable()) {
      List<String> attributes = new ArrayList<String>();
      attributes.add("cn");
      addAttribute(attributes, givenNameAttribute);
      addAttribute(attributes, surNameAttribute);
      addAttribute(attributes, localeAttribute);
      addAttribute(attributes, organizationAttribute);
      addAttribute(attributes, divisionAttribute);
      addAttribute(attributes, rolesAttribute);
      SearchRequest request = new SearchRequest(userDn, SearchScope.SUB,
          Filter.createEqualityFilter("objectclass", "person"),
          attributes.toArray(new String[attributes.size()]));
      return getUserInfo(user, connection.search(request));
    }
  } catch (LDAPException le) {
    String message = le.getMessage();
    if (!message.startsWith("Unable to bind as user ")) {
      log.error(le.getMessage(), le);
    }
  } catch (GeneralSecurityException gse) {
    log.error(gse.getMessage(), gse);
  } finally {
    if (null != connection) {
      connection.close();
    }
  }
  return null;  // not logged in
}

The information about the logged in user and the roles is built from the configured attributes in the getUserInfo method.

private UserInfo getUserInfo(String userId, SearchResult search) {
  if (search.getEntryCount() > 0) {
    SearchResultEntry entry = search.getSearchEntries().get(0);
    UserInfo result = new UserInfo();
    result.setUserId(userId);
    String name = entry.getAttributeValue(givenNameAttribute);
    String name2 = entry.getAttributeValue(surNameAttribute);
    if (null != name) {
      if (null != name2) {
        name += " " + name2;
      }
    } else {
      name = name2;
    }
    result.setUserName(name);
    result.setUserLocale(entry.getAttributeValue(localeAttribute));
    result.setUserOrganization(entry.getAttributeValue(organizationAttribute));
    result.setUserDivision(entry.getAttributeValue(divisionAttribute));
    result.setAuthorizations(getAuthorizations(entry));
    return result;
  }
  return null;
}

The full source code can be found here.

Categories: geomajas / GIS, java
Previous Entries
Next Entries
  • Recent Posts

    • Plug-ins and faces overview for Geomajas
    • Adding an admin user in JIRA with embedded database
    • The Geomajas project release releases back-end 1.10, three independent projects and 15 plug-ins
    • Spatial data in the enterprise BeJUG session
    • Ktunaxa Referral Management System
  • Recent Comments

    • Eric on delete windows service account
    • nandra indonesia on delete windows service account
    • joachim on CXF ws client, dynamic endpoint and loading WSDL from the classpath
    • TXMaster on CXF ws client, dynamic endpoint and loading WSDL from the classpath
    • TXMaster on CXF ws client, dynamic endpoint and loading WSDL from the classpath
  • Categories

    • architecture
    • competencies
    • equanda
    • geomajas / GIS
    • java
    • jboss
    • maven
    • semantics
    • tapestry5
    • ubuntu / debian / linux
    • Uncategorized
    • web development
    • web services
© trying to solve IT problems. Proudly Powered by WordPress | Nest Theme by YChong