So we had the opportunity to test OrenterNET with @cquest during the IGN Hackathon
In short,
the library is fast, very well documented, consumes very few ressources, and can even handle images without GPU.
Some learnings :
- don’t send full 3D images. This has been trained with crops. We can extract 4 images from a 360 scene and average the 3 closer locations
- Use 512px images, this is fast and this is the size of the image used in training sets
- 256 px gives wrong results. > 800px consumes too much ressources for no gain
- Using sequences is a lot better because it reduces the degrees of freedom in the context used
- We can train our own dataset and run efficiency benchmarks, but it requires good GPUs (we didn’t test )
- the code repo is kindof archived. Some pull request branches bring very interesting new features, but the repo should be revived or forked
- If the environnement has very symetric real objects (building, roads etc..), it will diverge a lot more. But less with sequences. In rural areas with very few OSM objects, the re-positioning can’t be as good as in dense areas
- It uses 40 classes of OSM ( buildings, roads, urban equipments etc..). Buildings provide more signal than the other classes. Trees could provide good signal but are not often mapped enough in OSM. We can freely configure the classes to use
- The model is open weight, trained in many european cities and very robust. There is no lockin in training a new model or in evaluation datasets. Very nice.
One example of a picture that has a GPS drift of ~ 10m , with bad verticality
Here is the map with the positioning probalities and in black the proposed location. It is spot on the bike lane, where all the other panoramax images are.
What to do next ?
We don’t have real plans, but we feel a real potential to propose better geographic positionning.
We could :
- run our own benchmark on RTK acquired pictures to qualify with panoramax specific high quality data.
- Do some sampling to evaluate if we can spot sequences with very bad positioning and check how good repositioning is.
Side note, I discovered the concept of High Dimensionality Neural Maps, wich encode vector maps object into a high dimensional raster that seem to be very usable in this area. Really promising
