blog entries created by Arthur Lutz

Brainomics - A management system for exploring and merging heterogeneous brain mapping data

2013/09/12 by Arthur Lutz

At OBHM 2013, the 19th Annual Meeting of the Organization for Human Brain Mapping, Logilab presented a poster which explains the work done using CubicWeb on brain imaging and genetics data in collaboration with INRIA, INSERM and the CEA during the Brainomics project co-financed by Agence nationale de la Rercherche.

http://www.cubicweb.org/file/3123353/raw/Screenshot%20from%202013-09-12%2010%3A27%3A27.png

You can download this poster and try the demo online.


We're going to PGDay France, the Postgresql Community conference

2013/06/11 by Arthur Lutz

A few people of the CubicWeb team are going to attend the French PostgreSQL community conference in Nantes (France) on the 13th of june.

http://www.cubicweb.org/file/2932005/raw/hdr_left.png

We're excited to learn more about the following topics that are relevant to CubicWeb's development and features :

https://www.pgday.fr/_media/pgfr2.png

Obviously we'll pay attention to all the talks during the day. If you're attending, we hope to see you there.


OpenData meets the Semantic Web at WOD2013

2013/06/10 by Arthur Lutz

With a few people from Logilab we went to the 2nd International Workshop on Open Data (WOD), on the 3rd of june.

Although the main focus was an academic take on OpenData, a lot of talks were related to the Semantic Web technologies and especially LinkedData.

http://www.logilab.org/file/144837/raw/banniere-wod2013.png

The full program (and papers) is on the following website. Here is a quick review of the things we though worth sharing.

  • privacy oriented ontologies : http://l2tap.org/
  • interesting automations done to suggest alignments when initial data is uploaded to an opendata website
  • some opendata platforms have built-in APIs to get files, one example is Socrata : http://dev.socrata.com/
  • some work is being done to scale processing of linked data in the cloud (did you know you could access ready available datasets in the Amazon cloud ? DBPedia for example )
  • the data stored in wikipedia can be a good source of vocabulary on certain machine learning tasks (and in the future, wikidata project)
  • there is an RDF extension to Google Refine (or OpenRefine), but we haven't managed to get it working out of the box,
  • WebSmatch uses morphological operators (erosion / dilation) to identify grids and zones in Excel Spreadsheets and then aligns column data on known reference values (e.g. country lists).

We naturally enjoyed the presentation made by Romain Wenz about http://data.bnf.fr with the unavoidable mention of Victor Hugo (and CubicWeb).

Thanks to the organizers of the conference and to the National French Library for hosting the event.


Links roundup from dotjs.eu

2012/12/05 by Arthur Lutz

A few people from Logilab attended the dotjs conference in Paris last week. The conference wasn't exactly what we expected, we were hoping for more technical talks. Nevertheless, some of the things we saw were quite interesting. Some of them could be relevant to CubicWeb.

http://www.cubicweb.org/file/2532779?vid=download

Here is a raw roundup of links collected last friday :


Follow up of IRI conference about Museums and the Web #museoweb

2012/04/12 by Arthur Lutz

I attented the conference organised by IRI in a series of conferences about "Muséologie, muséographie et nouvelles formes d’adresse au public" (hashtag #museoweb). This particular occurence was about "Le Web devient audiovisuel" (the web is also audio and video content). Here are a few notes and links we gathered. The event was organised by Alexandre Monnin @aamonnz.

http://polemictweet.com/2011-2012-museo-audiovisuel/images/slide4_museo_fr.png

Yves Raimond from the BBC

Yves Raimond @moustaki made a presentation about his work at the BBC around semantic web technologies and speech recognition over large quantities of digitized archives. Parts of the BCC web sites use semantic web data as the database and do mashups with external sources of data (musicbrainz, dbpedia, wikipedia). For example Tom Waits has an html web page : http://www.bbc.co.uk/music/artists/c3aeb863-7b26-4388-94e8-5a240f2be21b add .rdf at the end of the URL http://www.bbc.co.uk/music/artists/c3aeb863-7b26-4388-94e8-5a240f2be21b.rdf

He also made an introduction about the ABC-IP The Automatic Broadcast Content Interlinking Project and the Kiwi-API project that uses CMU Sphinx on Amazon Web Services to process large quantities of archives. A screenshot of Kiwi-API is shown on the BBC R&D blog. The code should be open sourced soon and should appear on the BBC R&D github page.

Following his presentation, the question was asked if using Wikipedia content on an institutional web site would be possible in France, I pointed to the use of Wikipedia on http://data.bnf.fr , for example at the bottom of the Victor Hugo page.

Raphaël Troncy about Media Fragments

Raphaël Troncy @rtroncy made a presentation about "Media Fragments" which will enable sharing parts of a video on the web. Two major features : the sharing of specific extracts and the optimization of bandwith use when streaming the extract (usefull for mobile devices for example). It is a W3C working draft : http://www.w3.org/TR/media-frags-reqs/. Here are a few links of demos and players :

Part of the presentation was about the ACAV project done jointly with Dailymotion : http://www.capdigital.com/projet-acav/

The slides of his presentation are available here : http://www.slideshare.net/troncy/addressing-and-annotating-multimedia-fragments

IRI presentation

Vincent Puig @vincentpuig and Raphaël Velt @raphv made a presentation of various projects led by IRI :

http://www.iri.centrepompidou.fr/wp-content/themes/IRI-Theme/images/logo-iri-petit_fr_fr.png

Final words

The technologies seen during this conference are often related to semantic web technologies or at least web standards. Some of the visualizations are quite impressive and could mean new uses of the Web and an inspiration for CubicWeb projects.

A few of the people present at the conference will be attending or presenting talks at SemWeb.Pro which will take place in Paris on the 2nd and 3rd of may 2012.


CubicWeb Sprint report for the "Benchmarks" team

2012/02/17 by Arthur Lutz

One team during the CubicWeb sprint looked at issues around monitoring benchmark values for CubicWeb development. This is a huge task, so we tried to stay focused on a few aspects:

  • production reponse times (using tools such as smokeping and munin)
  • response times of test executions in continuous integration tests
  • response times of test instances runinng in continuous integration

We looked at using cpu.clock() instead of cpu.time() in the xunit files that report test results so as to be a bit more independent of the load of the machine (but subprocesses won't be counted for).

Graphing test times in hudson/jenkins already exists (/job/PROJECT/BUILDID/testReport/history/?) and can also be graphed by TestClass and by individual test. What is missing so far is a specific dashboard were one could select the significant graphs to look at.

By the end of the first day we had a "lorem ipsum" test instance that is created on the fly on each hudson/jenkins build and a jmeter bench running on it, it's results processed by the performance plugin.

http://www.cubicweb.org/file/2184036?vid=download

By the end of the second day we had some visualisation of existing data collected by apycot using jqplot javascript visulation (cubicweb-jqplot):

http://www.cubicweb.org/file/2184035?vid=download

By the end of the sprint, we got patches submitted for the following cubes :

  • apycot
  • cubicweb-jqplot
  • the original jqplot library (update : patch accepted a few days later)

On the last hour of the sprint, since we had a "lorem ipsum" test application running each time the tests went through the continuous integration, we hacked up a proof of concept to get automatic screenshots of this temporary test application. So far, we get screenshots for firefox only, but it opens up possibilities for other browsers. Inspiration could be drawn from https://browsershots.org/


HTML5 features presented at Paris Web 2010 by Paul Rouget

2010/10/19 by Arthur Lutz

While at Paris Web 2010 we were all impressed by the presentation and demos by Paul Rouget on HTML5 (tech evangelist must be a hard job!). Here is my take and a few URLs on the things that were presented.

http://hacks.mozilla.org/wp-content/themes/Hacks2010/img/mozilla.png
  • Websockets with persistent connections between the server and the browser. That way you can avoid pulling information every 5 seconds, the server can tell the web page a new info is available. The immediate uses we have for this are :
    • realtime feed display
    • jabber web chat rooms
    • in cubicweb's forge : new comment indication on a ticket
    • in cubicweb in general : notification that the edited element has been openned by another user (instead of a lock mechanism)
    • real time collaborative editing (etherpad style functionality)
  • File upload demo : http://demos.hacks.mozilla.org/openweb/uploadingFiles/
  • File EXIF extraction, client side resize or geolocalisation http://demos.hacks.mozilla.org/openweb/FileAPI/ . That could be very cool for things such as resizing an image before it is sent to the server (you know, for your mother who doesn't know how to resize that 2 Mbytes photo before sending it to the site). Reference : https://developer.mozilla.org/en/Using_files_from_web_applications
  • Using File IO, you can do some heavy Drag'n'drop from your computer to your browser directly in the browser (yes, you can get rid of that nasty java applet). Apparently Google implemented in Chromium a non-standard drag'n'drop the other way around : from the web app to your desktop, which could be cool as well.
http://farm5.static.flickr.com/4147/5085028912_173337f0ba.jpg
  • XHR - XMLHttpRequest. Usually this type of requests is not possible cross-domain. Now they will be (with an authorization mechanism). That way, you will be able to post and control websites from the page in your browser.
  • Audio Data API : you can now access & modify audio files directly in your browser (before uploading them server side). This makes me think of the first time I realized people where implementing traditionally "heavy" applications (photo editing, music editing, even movie edition) in web applications. I was (and still am) very surprised and skeptic, but this kind of evolution makes me believe that there can be a day when you don't even need to send massive files to the server to edit them.
http://farm1.static.flickr.com/191/513636061_98d07f7966_t.jpg

Admittedly, you probably need to see the thrilling presentation and demos to be tempted to go and dip into these technologies. Reading the documentation will probably not encourage you to go and code some cool new features.

One of the things that the audience commented about at the end of the presentation is that there was still a huge lack of "authoring tools" for HTML5. For some coders that never leave vim or emacs, this is heresy, but we have to admit that the adoption of flash and silverlight (apparently) is very much driven by simple click'n'program tools.

http://www.mozilla.org/images/minefield_168.png

During the presentation, I used a Chrome 6 that I had lying around on my Ubuntu, but by the end of the presentation I had installed Firefox4 using the mozilla PPA

sudo add-apt-repository ppa:ubuntu-mozilla-daily/ppa
sudo apt-get update
sudo apt-get -uVf install firefox-4.0

The PPA version keeps config files separate so you can easily switch between your "standard" Firefox3 profile and the cutting edge Firefox4 (obviously the big downside is not having all your cool extensions).

The only thing missing from the presentation was the code... a request I hope Paul will grant to the community (a bunch of tweets about that followed the presentation).


CubicWeb presentation at the JDLL (Lyon)

2010/10/07 by Arthur Lutz

For the "Journées Du Logiciel Libre (JDLL)" in Lyon which will take place the 14th, 15th et 16th of octobre 2010, we will be presenting the semantic side of CubicWeb on Friday 15th. There will be a talk and a tutorial. Details can be found here and there.

If you're around, come and see us!

http://www.jdll.org/sites/default/files/banniere.png

CubicWeb sprint in Paris about js and css

2010/04/29 by Arthur Lutz

Logilab is once again hosting a sprint around CubicWeb - 5 days in our Paris offices.

The general focus will be around javascript & css :

http://www.iconarchive.com/icons/enhancedlabs/lha-objects/128/Filetype-CSS-icon.pnghttp://codesnip.net/wp-content/uploads/javascript.png
  • easily change the style of an application
  • handling of bundles merging javascript and css
  • have a clean javascript API, documented and tested
  • have documentation about the css & javascript parts in the cubicweb book

This sprint is taking place from thursday the 29th of April 2010 to the 5th of may 2010 (weekend is off limits - the offices will be closed). You are more than welcome to come along and help out, contribute, or just pair program with someone. Coming only for a day, or an afternoon is fine too... Network resources will be available for those bringing laptops.

Address : 104 Boulevard Auguste-Blanqui, Paris. Ring "Logilab".

Metro : St Jacques or Corvisart (Glacière is closest, but will be closed from monday onwards)

Contact : http://www.logilab.fr/contact

Dates : 29/04/2010 to 30/04/2010 and 03/05/2010 to 05/05/2010


Migrating cubicweb instances - benefits from a distributed architecture

2010/04/22 by Arthur Lutz

Aim : do the migration for N cubicweb instances hosted on a server to another with no downtime.

Prerequisites : have an explicit definition of the database host (not default or localhost). In our case, the database is hosted on another host. You are not migrating your pyro server. You are not using multisource (more documentation on that soon).

Steps :

  1. on new machine : install your environment (pseudocode)

    apt-get install cubicweb cubicweb-applications apache2
    
  2. on old machine : copy your cubicweb and apache configuration to the new machine

    scp /etc/cubicweb.d/ newmachine:/etc/cubicweb.d/
    scp /etc/apache2/sites-available/ newmachine:/etc/apache2/sites-available/
    
  3. on new machine : give new ids to pyro registration so the new instances can register

    cd /etc/cubicweb.d/ ; sed -i.bck 's/^pyro-instance-id=.*$/\02/' */all-in-one.conf
    
  4. on new machine : start your instances

    cubicweb start
    
  5. on new machine : enable sites and modules for apache and start it, test it using by modifying your /etc/host file.

  6. change dns entry from your oldmachine to newmachine

  7. shutdown your old machine (if it doesn't host other services or your database)

  8. That's it.

Possible enhancements : use right from the start a pound server behind your apache, that way you can add backends and smoothily migrate by shuting down backends that pound will take into account.

http://www.cubicweb.org/image/893561?vid=download