Picture a busy dock in a large port anywhere in the world. A large seagoing vessel is ready to be refueled… but then the fuel hose isn't properly attached. Instead of flowing into the ship, oil pours into the water. Before it can be contained, the slick drifts across a dock lined with moored vessels and out towards the open sea. This oil spill prevents ships from leaving the port, causes significant environmental pollution, and will cost the port to clean up.
Could the port’s cameras have raised the alarm sooner? That's a question the Port of Antwerp-Bruges and Milestone Hafnia are tackling together. This post is an early look at that work: what we're testing, what makes it challenging, what we're learning, and what’s next in our collaboration.
Port of Antwerp-Bruges, in northern Belgium, is Europe's second-largest commercial port. It connects the North Sea to the heart of Europe. Every year, it handles hundreds of millions of metric tons of cargo, with thousands of vessels moving through its docks, terminals and waterways.
Hafnia, Milestone Systems' vision AI platform, is working with the port to explore whether vision-language models (VLMs) — AI models that can process video and describe what's happening in plain language — can support safety and security operations in an environment this large and complex. It's a hands-on collaboration: we test ideas together, learn from what the results show, and share what we find along the way.
Together, we're looking at applying VLM technology to use cases around the port, including:
- Camera scene quality checks
- Bridge safety alert explanation
- Oil spill detection
This post focuses on oil spill detection because it's a uniquely difficult problem, and one with huge potential benefits.
Oil spills are extremely difficult to detect using computer vision technology because they rarely look the same twice. Their appearance shifts with lighting, reflections, weather, camera angle, the type of oil, and the position of the sun. Adding further complexity is the fact that plenty of harmless things on the water look like oil: silt, floating debris, mist, or cloud reflections. That makes false positives a constant challenge. Universities and commercial experts have been working on this problem for years, and a highly-accurate solution has yet to appear.
Then there's the data problem. Oil spills are thankfully rare, but that means there isn't much real footage that can be used to train computer vision models for more accurate performance.
Deploying a vison AI agent that is contextually aware should help prevent potential environmental impacts or costly downtime.
We need footage of oil spills to train models, but this footage is rare — and would be environmentally catastrophic to recreate in the real world. This is where synthetic data comes in. The teams at Port of Antwerp-Bruges and Hafnia are using anonymized port footage that has been augmented with synthetic oil-spill sequences, generated using NVIDIA Cosmos.
These synthetic data sets let Milestone Hafnia train and test VLMs against difficult cases like oil spills and the look-alikes that trigger false alarms. In practice, synthetic clips have been most valuable as a stress test. The team deliberately generated "traps": expanding mud patches, calm water with shifting reflections, foam and propeller wash, even distractions like vehicles. Each one probes a different way the model could be fooled. Real-world port footage remains the ultimate measurement for how well the model performs.
As with many of Milestone’s generative AI products, this VLM is built as an early-warning layer to alert operations teams when intervention is needed.
Here’s how it works:
- When the model detects something that might be a spill, it describes in plain text what the camera has captured.
- An operator can read the plain text description, which includes the camera’s location and angle, and scan the video footage themselves to confirm what the VLM has detected.
- Then, they can decide whether the situation is serious enough to send someone to the scene or deploy a drone for more reliable confirmation.
This approach brings together the port's operational knowledge and camera infrastructure with Hafnia's vision AI workflows.
The collaboration is still early, and first round of tests on real footage from the port's fixed cameras and drones has already taught us a lot. The tests used NVIDIA Cosmos 3 Super as the underlying vision-language model. We tested it on 119 human-verified, 30-second clips: 93 showing oil from three spill events, and 26 of oil-free water that included look-alikes such as wakes and stirred-up silt.
What’s working:
- One sentence made a big difference: The look-alike the port's experts worried about most was mud and sand stirred up from the harbor bottom, which can discolor the water much like a spill. The fix was surprisingly simple: telling the model that oil looks glossy and mirror-like on the surface, while stirred sediment looks matte and opaque. With that guidance, the model correctly rejected all eight sediment plumes in the test footage.
- The model shows its reasoning: Instead of a bare yes or no, it explains what it sees, so operators can quickly judge whether to act.
- Answers are consistent: Running each clip six times returned the same verdict every time, an important foundation for trust.
Here's how the model described one clip of a real spill at the port:
A few things we're still working on:
- Calm water that mirrors the sky: On a still day, the harbor can reflect the sky with the same glossy sheen that helps separate oil from mud. We saw this confusion in both real and synthetic footage, which gives the team a clear, specific next problem to solve.
- Faint slicks far from the camera: When a thin slick is a long way from a fixed camera, there simply may not be enough detail in the image. That's a limit of camera resolution rather than the model's understanding, and it's where camera placement and drones come in.
With a promising first round behind us, the next step is broader evidence. Over the coming year, we'll continue working through the four use cases with Port of Antwerp-Bruges and report back on what we find. For oil spills, that means testing the model to tell a glossy slick from a calm, sky-mirroring harbor, and testing it on more camera views and more spill events.
As Pieter Van Bouwel put it, oil spill detection is the “stress test” of computer vision. Spotting a spill sooner isn't just a feat of technical ingenuity, it also limits environmental damage, keeps ships moving, and could save ports around the world millions in cleanup costs. Every spill caught early is a smaller spill to clean up. And if this “stress test” problem can be cracked, many other computer vision challenges come within reach. We look forward to continued collaboration with Port of Antwerp-Bruges and other organizations around the world.
