This is very interesting !
I tried to use cubemaps too for this a few months ago.
Turns out there has already been some research ACCV 2018 Deep Upright Adjustment of 360 Panoramas Using Multiple Roll Estimations. My original approach extracts 4 sides/views and calculates the angle of each of them then averages them. The approach from the publication uses narrow 36 views with FOV of 60 degrees at 10 degrees apart, predicts the angles then map the ‘waviness’ of the tilt in the photo, exclude the outliers and finally use the results to level the photo.
It does significantly help with the tree/wall too close to the camera problem where those angle predictions differ from reality and can be excluded as outliers. The research publication doesn’t have code available, so I re-used what I already worked on. The current model which I still haven’t yet trained is producing too many outliers needed for sufficient levelling. It does mean almost all photos are level to within several degrees. I must be expecting too much.
but…
I started looking into newer projects. From 2024, there is cvg/GeoCalib: GeoCalib: Learning Single-image Calibration with Geometric Optimization (ECCV 2024) which combines deep learning, visual cues and geometric features. Even though it was trained on panoramic images, it isn’t built to input panoramic images. It was very easy to install and once again, I took the 36-view idea and tested with random Panoramax photos on bicycle. The results are very exciting, about 97% were suitably upright and without needing to do my own training! It did very well with all of the challenging photos from my first attempt.
Just as I was about to be done for the day, I saw one more very recent 2026 publication 360Anything: Geometry-Free Lifting of Images and Videos to 360°. They also use GeoCalib to upright panoramas by using only 8 views.
It would be very nice not needing to feel dizzy browsing Panoramax!
5 Likes
Updated proof of concept replacing it with the Panoramax viewer: https://leveltest.seen.one/
Some patience will be needed; i’m using an old laptop with only the CPU to calculates the corrections on the fly. It will feel very slow to load per 360° photo because it takes about 10 seconds to process (fast Nvidia GPU takes about 2 seconds). Ideally the corrections would have been pre-calculated, but for this demo, this should show great benefits to the bicycle mappers/viewers!
Time for an actual horizon correction API like the blurring API?
4 Likes
Very nice job ! The result is impressive !
2 Likes
On non 360° picture, a common error is full rotations (+/- 90° or event 180°).
I’ve quickly created a dataset and trained a model to detect them by rotating pictures myself :
It is a classifier which works on very small images (224x224 pixels) meaning we can run it on thumbnails and it takes around 10ms for an inference.
After fine-tuning, a YOLO26 “small” gives 99.7% accuracy and uses only around 500MB of VRAM.
3 Likes
I’ve tested with Geocalib on 4 reprojected cube faces.
Original :
First pass :
Second pass :
A second pass seems necessary to get a better result when the pitch+roll angles on the first pass are high (above 10° ?). Most of the time this second pass increases the first pass angles.
I’ve tested with the full and low resolution of the same pictures (16Mpx vs 2Mpx / 2048 pixels wide) and the results are similar, but the processing time is of course better.
On low res, one pass takes less than a second, two passes less than 2 seconds.
Geocalib inference uses less than 1GB of GPU VRAM.
Geocalib allows to provide the focal length when known, and here we know it because we choose it when generating cube faces from the equirectangular picture.
I’ve used 30° and tested other value with similar results.
The generated faces are 256x256 pixels :
More tests should be done on less urban pictures…
4 Likes
Ça a l’air très bien tout ça ! Dans quelle mesure est-ce industrialisable ?
Vu que ça consomme quand même pas mal de ressources, ce n’est pas un traitement que je verrai appliqué systématiquement, mais plutôt à la demande, soit du contributeur au moment de l’envoi ou après, soit quand quelqu’un signalerai des photos à remettre à l’horizontale.
Une possibilité serait de s’appuyer sur les tags sémantiques:
- on ajoute un tag indiquant que c’est de travers (
fixme:pitch=yeset/oufixme:roll=yes) - un bot récupère les images qui ont ce tag et le remplace par un tag avec les valeurs de correction (pitch/roll)
- la visionneuse tiendrait compte du tag de correction à l’affichage et pas que dans les EXIF, comme on le fait pour l’orientation.
La correction manuelle parait une option envisageable, en effet. On peut imaginer une review collaborative des séquences existantes.
Et pourquoi pas corriger seulement la première photo d’une séquence (ou plutôt une des premières photos, pour éviter la première photo parfois ratée). Si la correction apparaît comme significative, alors et seulement alors on retraite toute la séquence.
Il faudrait aussi ajouter “photos de travers” dans les choix proposés au signalement d’une photo, comme il y a “photo surfloutée”.
J’ai signalé tout un lot de photos “pivotées” en cochant “autre” la semaine passée, et en détaillant chaque fois image de travers, etc.
Le grand “public”/commun des mortels non tech aura le réflexe de signaler la photo plutôt que de se lancer dans des tags.
2 Likes
In addition, I was thinking we can use existing tags/metadata to identify sequences that likely need correction like transport=bike and GoPro Max. Professional setups would less likely need correction like transport=car and imajbox/Ladybug/Trimble 360 cameras.
2 Likes
Yes and if there is already XMP tags with pitch/roll info, we can consider they are ok.
I see the manual tagging + bot solution as a first step, then if we can do more it could be nice after some feedback of the first step results.
2 Likes
On pourrait aussi intégrer ça côté poste utilisateur soit au moment du prétraitement (lancement d’un docker/app image) ou à postériori (quand l’API permettra de modifier les exifs).
Est ce que vous avez des tests sur des routes/voies inclinées (+5%) ? Dans les corrections manuelles que je fais c’est pas évident de trouver le bon référentiel avec une voie qui n’est pas parallèle à l’horizon.
Le même cas avec des pins d’alep (ces grands arbres qui ne sont jamais verticaux) semble un bon challenge pour pousser la correction dans ses retranchements 
2 Likes
Oui, on peut très bien le faire avant versement, mais pour beaucoup de monde ce sera quand même pas évident et cela dépend beaucoup de comment on fait les prises de vues.
Effectivement il y a sûrement des cas moins évidents… il faut trouver des photos dans ce genre de situation pour faire des tests.
J’ai déjà rajouté une série d’images en forêt à mon lot de test dans ce but.
2 Likes
Q&D way to fix the images EXIFs with Docker based on @seen-one work
1 Like
I was thinking of doing something similar 
deep-image-orientation-angle-detection was the older project that I initially tested. GeoCalib is more accurate and is what is currently being used on leveltest.seen.one and what cquest is testing with their development.
Similar proof of concept code based on the initial repo I posted can be seen below. Sorry I should have posted the link earlier.
1 Like
I’m doing the first test of a bot which :
- search for pictures with the
fixme:roll=yestag - download the low resolution of found pictures
- computes the pitch/roll to apply
- updates the picth/roll metadata using Panoramax API
- updates the
fixme:roll=yesintofixme:roll=fixed(WIP)
Here is the first one processed like that:
Before the fix:
5 Likes
More before/after samples done by the bot with Geocalib:
It is currently running on some full sequences of several hundreds/thousands of pictures.
8 Likes
Some heuristic added to the bot… after fixing a picture:
- check if the pitch/roll are above 5°
- if so, fix the next and previous pictures in the sequence as long as they have more than 5° pitch/roll
- if more than 10 pictures have been fixed in the sequence, fix the whole sequence
This allows a single fixme:roll=yes added manually to start fixing many pictures.
Time to process one picture is around 1.3s on my workstation, which includes:
- download the low-res version,
- reproject 4 faces,
- predict roll on each of them with 1 or 2 passes (around 100ms per model inference on my 4090),
- send the resulting metadata to the API
Currently more than 33k have been processed.