Skip to main content

Going to the Geonetwork CodeSprint in Bolsena

The week of the 4th to 8th of June 2018 was the 11th annual Geonetwork Codesprint in Bolsena, coordinated by GeoCat. As in previous years, the Geonetwork developer team took over an ex-convent, just outside the small town of Bolsena, on the side of the lake of the same name, about 135km north of Rome.

View Larger Map

At Astun we've always had a firm belief in contributing back something to the open source software that we use, be that by sponsoring or providing enhancements, fixing bugs, or getting involved with conferences and hackathons. Due to the work we've been doing supporting and extending Geonetwork for our local Government INSPIRE metadata portal, for the Scottish Spatial Data Infrastructure portal, and with DEFRA, this event seemed like a good opportunity to meet the core developers face to face, to learn something new, and perhaps assist with some development.

I went along with no fixed expectations of what would be involved, or what I could contribute. I consider myself to be a non-coder, although I know my way around the Geonetwork repository on GitHub well enough to compile and deploy the code from source and to debug issues. Furthermore, these days the Geonetwork code is only part of the story! The metadata is stored in a back-end database, generally PostgreSQL, and then the search results are indexed in ElasticSearch, with Kibana providing dashboarding capabilities (more on this later).

So what does a non-coder do at a codesprint? Plenty, as it happens!

Day one was mainly about orientation. There was a "state of the nation" talk going through the metrics of the project (the number of contributors, the number of bugs reported and fixed, and so on). This was followed by some ideas as to what might be achievable in the week. These basically broke down to the following over-arching themes:

  1. Remove the historic dependencies on Lucene for searching and move entirely over to ElasticSearch 
  2. Refactor the release and branching workflow in the Github repository so that new features were added to the correct (future) release and not to the current stable branch
  3. Test a new script for generating new metadata schemas or profiles 
  4. Squash as many bugs and merge as many pull requests as possible! 

Over the next few days we worked through these main themes, punctuated by lovely home-cooked meals sat outside in the convent garden, some multi-lingual karaoke, an afternoon wandering through the medieval streets of Bolsena, and an evening at the National Museum in Rome.

The move to ElasticSearch was a big step in Geonetwork 3.4, and has felt to me like a black art. There's a lot of new technology to learn (shards, index patterns, mappings, clusters, nodes...) and a new syntax for querying. I was concerned that end-users might struggle to reproduce queries that they had created in previous versions of the software without a lot of additional support. So while the developers were working on the code, I created some documentation on ElasticSearch for new Geonetwork admins, and also investigated a plugin that provides a more familiar SQL interface to the index. The latter needs a lot of work to be useful, I think, but it could be handy for people getting used to the new technology, like me for example!

Refactoring the release and branching workflow sounds very dull, but was a very important step in improving the quality and sustainability of releases. In a nutshell it means that new features are introduced in new major releases of the code rather than appearing unannounced in a patch. This required a lot of work on the part of the developers, and it's not something I could really help with, other than by supporting it as a Really Good Idea.

One thing I was very excited to see was the new script for generating a metadata schema or profile.
At Astun we've contributed towards the development of the Gemini 2.2 metadata profile for Geonetwork, and have also built an extended metadata profile for the Environment Agency to allow them to store some additional metadata elements, such as an Approval for Access code, and also to include additional validation rules. This process is quite time-consuming, so being able to automate at least the basic process will be extremely handy, particularly when we come to work on a Gemini 2.3 plugin later this year.

Bug squashing and pull request acceptance was fun, and rather satisfying. Many bugs had been superseded by later versions of the code (including some of my own), or had not been commented on for over a year. These could all be closed. Others needed tagging to ensure they were matched to the correct release, or needed more information before they could be reproduced. I was made a contributor to the project so I could assist with this. I have plans to start creating bugs for unfinished documentation now I can tag them as such. Documentation bugs are a really easy way for beginners to contribute to a project, as well as being extremely valuable to the developers.

All in all, it was a great week, and I would definitely argue that non-coders can contribute to, and get a lot out of codesprints. While we worked hard, there's something about being in lovely surroundings, with nice weather and good company to make it feel a lot less like work!


Popular posts from this blog

Reflections on FOSS4G UK Edinburgh 2019

Six members of the Astun team recently headed en mass to the Free and Open Source Software for Geo (FOSS4G) UK conference in Edinburgh - and we came back with a 7th!

FOSS4G is a great forum to find out all the latest on what is going on in the OSGeo community and to spread the word via talks and workshops about some of the exciting work we’ve been involved in. Check out the post to find out about our own personal reflections of the event … and find out who our latest recruit is in the final post!

Dan The FOSS4G UK conference just seems to get better and better each year, and this year was no exception. As the baton gets passed annually to a new organising team, the advice and guidance on running a great event goes with it. This year’s team led by co-chairs Tom Armitage and Ross McDonald did an amazing job. The venue was breathtaking, the agenda was packed with outstanding talks and workshops and the atmosphere both during the day and in the evening social just had a great buzz to it. …

Wychavon District, Malvern Hills District, and Worcester City councils sponsor improvements to the Gemini metadata plugin for Geonetwork

Astun have been working with Wychavon District, Malvern Hills District and Worcester City councils on some improvements to the Gemini 2.2 Metadata Plugin for Geonetwork.

The councils approached us at one of our User Group events in 2018 for some assistance with their joint data and metadata publishing workflow. In this workflow, data is published as WMS and WFS using Geoserver. Metadata in the WMS and WFS responses are harvested by Geonetwork to create metadata records, which are then published to The goal was to create fully valid Gemini 2.2 metadata directly from Geoserver, without the need for editing the records in Geonetwork. We worked with the councils to establish that the Geoserver INSPIRE plugin and built-in metadata tools met most of that requirement, but that some elements were either incorrectly added or missing entirely when the metadata was harvested into Geonetwork.

Astun have enhanced the Gemini 2.2 metadata plugin for Geonetwork to improve it's WMS an…

FOSS4G UK Edinburgh 2019

This year, Astun staff are giving 4 presentations and 1 workshop at FOSS4G UK Edinburgh to share with the community what we are passionate about as individuals, and as an organisation. At Astun we believe that we are the 'Experts in Place', but to live up to this claim we must do everything but stay 'in place' by constantly evolving what works, and revolutionising what doesn't. With that in mind, here's a preview of the presentations we're giving next month:
Matt Walker; OpenLayers Workshop The OpenLayers (OL) Workshop will guide attendees through the official Workshop material, providing a comprehensive overview of OL as a web mapping solution. The workshop format follows a series of modules covering everything from the basics of creating an OL map, through to specific functionality such as handling vector data, and building content for consumption on mobile devices.

We will work through as many of these modules as we have time for! The workshop material is…