Raleigh Turns Drones and GeoAI into a Living Digital Twin of the City

Raleigh Turns Drones and GeoAI into a Living Digital Twin of the City

Raleigh is building a digital representation of the city that is designed to change whenever the city itself does. Drone surveys of construction sites, infrastructure, storm damage and public spaces can be processed into new imagery and three-dimensional models, fed back into the city’s geographic information system and incorporated into an increasingly detailed digital twin.

Cities have been creating 3D visualisations and GIS datasets for years, but their usefulness deteriorates when the physical environment moves faster than the information describing it. Roads are resurfaced, buildings appear, utilities are moved, construction sites advance and storms can alter parts of the landscape in hours.

Raleigh’s approach brings reality capture much closer to everyday municipal operations. Drones provide repeatable observations of the physical city, GIS supplies the authoritative spatial framework, and Esri’s mapping and processing technology turns those observations into information that can be compared, shared and increasingly analysed using geospatial artificial intelligence. The result is beginning to look less like a conventional city model and more like a continuously maintained record of the built environment.

Briefing

• Raleigh is developing a living digital twin combining drone imagery, 3D reality mapping and authoritative municipal GIS data.
• Drone surveys can update the model following construction activity, infrastructure inspections, major events and storm damage.
• Imagery from flights can be processed into True Orthos and 3D meshes before being published through the city’s ArcGIS environment.
• Raleigh is experimenting with GeoAI to identify pavement markings automatically from captured road imagery.
• The city intends to extend this work to additional features including lane markings and pedestrian crosswalks.

Keeping the Model Current

The difficult part of a city-scale digital twin is not necessarily creating the first representation. It is maintaining a sufficiently current representation to make the information operationally useful.

Raleigh’s continuing growth makes that problem particularly visible. New development, infrastructure work and changes to public spaces create a physical environment that does not remain static for long. The city already maintains a GIS-based construction tracker covering municipal and state projects as well as permitted private work within the public right-of-way, demonstrating the volume of activity that has to be understood spatially across an urban area.

Drone operations provide another layer of observation. Instead of commissioning a major survey whenever updated information is required, municipal teams can capture particular locations as part of routine work. Flights can document construction progress, inspect infrastructure, record conditions around major events or capture damage following severe weather. Once collected, the imagery can move through a common geospatial workflow rather than remaining isolated within individual projects or departments.

Raleigh has been using Esri’s ArcGIS Flight for mission planning, with existing GIS datasets providing contextual information for operations in complex urban environments. Following a flight, imagery can be processed through Site Scan into True Orthos and 3D meshes and published through ArcGIS Online, where it becomes part of the wider digital representation of the city.

The city’s new city hall project provided a practical example. Drone flights captured the changing construction site in a dense downtown environment, while GIS information helped teams plan operations around surrounding constraints. The resulting imagery and 3D data could then be incorporated into the city’s digital environment rather than becoming a standalone collection of progress photographs.

The workflow forms a feedback loop. GIS helps organise the flight, the flight records the physical environment, processing converts those observations into spatial information, and that information returns to the GIS.

From Mapping to City Operations

Storm response is one application. Wide-area damage can be difficult to assess quickly from ground level, particularly when roads, trees, structures or utilities have been affected across several locations. Drone flights can provide an initial view of conditions while establishing a spatial record for subsequent assessment.

Infrastructure inspection presents a different problem. Bridges, roofs, pool enclosures and other structures can contain areas that are awkward, expensive or potentially hazardous to inspect conventionally. A drone does not eliminate the need for engineering inspection, but it can provide rapid visual information and help establish where closer investigation may be required.

Construction monitoring is particularly well suited to repeated capture because the value lies in comparison over time. A single survey records a moment, while a sequence begins to reveal change. As those sequences accumulate within the same spatial environment, the digital twin can provide a historical as well as a current representation of a project or location.

Planners can compare proposed development with its surroundings, project teams can follow construction progress, and city staff can work from a common geographical view rather than separate drawings, photographs and departmental datasets. Three-dimensional models can also communicate spatial relationships that are difficult to convey through conventional plans, particularly to people who do not routinely work with engineering drawings.

GeoAI and Pavement Markings

Raleigh’s experiments with geospatial AI take the programme beyond visualisation.

The city has begun developing a workflow for detecting pavement markings in captured imagery. Staff analyse pixels and ratios within images to classify arrow markings and use labelled examples to create a training dataset. A trained model can then be used to identify similar markings across larger quantities of road imagery.

The initial application is deliberately narrow. Arrows are recognisable physical features with a defined location and geometry, making them a useful starting point for testing whether information can be extracted reliably from imagery rather than manually identified each time. Raleigh plans to expand the approach to other pavement features including lane markings and crosswalks.

This begins to change the role of the imagery. Instead of serving solely as something for a person to inspect, it becomes a source from which infrastructure information can potentially be extracted automatically.

A flight undertaken for one purpose could consequently produce information useful for another. Imagery collected around a construction project might also contain road markings, kerbs, vegetation, signs or other infrastructure features. Provided suitable models can identify them reliably, the same source imagery could support multiple datasets rather than serving only the task that originally justified the flight.

The progression is from capturing the city to recognising what is within it. Beyond that lies a more demanding possibility: identifying what has changed between successive observations.

Repeated imagery of the same roads, structures and public spaces provides the raw material for comparison, while GeoAI offers tools for identifying features within those images. Combining the two could eventually allow particular changes to be flagged for attention rather than relying entirely on somebody to discover them manually.

That remains a potential development rather than an established Raleigh capability. Reliable change detection would require consistent data, suitable training, validation and sufficient confidence in automated results before they could support consequential engineering or maintenance decisions. Raleigh’s pavement-marking work is a contained application, but it provides a practical starting point for exploring how automated interpretation might sit alongside routine reality capture.

A Shared Spatial Record

Raleigh’s programme is spreading beyond the GIS team. Parks staff can use drone and geospatial information for managing green spaces and maintenance, while transportation and water departments can draw on the same underlying capabilities for planning and inspections. Sharing hardware, expertise and processing workflows across departments makes a citywide programme considerably different from individual teams buying drones for isolated applications.

A common spatial environment also addresses a longstanding problem in municipal information management. Roads, drainage systems, development projects, parks, structures and utilities may be administered separately, but they occupy the same physical city. A living digital twin provides a potential common reference point between those functions by connecting authoritative records with sufficiently recent evidence of what is actually on the ground.

“Our living digital twin informs city operations, helps us better predict and understand impacts across the city, and enables us to better serve our residents. With our large and growing footprint, it’s important to capture updated data about the city whenever the opportunity arises,” said Jim Alberque, GIS and Emerging Technology Manager at the City of Raleigh. “Esri’s integrated technology has made us much more agile, from planning flights and collecting data to detecting and analyzing changes in the imagery we capture.”

The city is effectively treating opportunities to observe the built environment as opportunities to improve the digital record describing it.

Digital Twins Become Operational

Digital twins have sometimes occupied an uncomfortable position in infrastructure technology, ranging from sophisticated engineering systems to attractive 3D visualisations. Their practical value depends heavily on what information sits behind the model, how current that information is and whether operational decisions can actually be made from it.

Raleigh provides a more incremental model. Authoritative geographic records, drone mission planning, repeatable reality capture, photogrammetric processing, 3D models and automated feature recognition are being connected around existing municipal work rather than assembled as a separate smart-city showcase.

The more interesting stage will come if those systems can reliably identify change. A municipal digital twin capable of comparing successive observations and directing attention towards altered assets or conditions would begin to move beyond showing staff what the city looks like. It could help them decide where to look next.

That possibility depends on dependable data, disciplined capture procedures and sufficient confidence in automated detection. The technology also has to fit the less glamorous realities of municipal work: inspections, maintenance, construction, storm response and keeping records current.

For Raleigh, keeping the virtual city useful therefore depends on something very physical: repeatedly going out and looking at the real one.

Key Industry Questions

1. What is Raleigh’s living digital twin? It is an evolving digital representation of Raleigh that combines authoritative GIS information with updated reality-capture data, including drone imagery and three-dimensional models.
2. How are drones used in the digital twin? Drone missions capture current conditions around construction projects, infrastructure, public spaces, major events and areas affected by storms. Processed imagery can then be incorporated into Raleigh’s geospatial environment.
3. What happens to the imagery after a drone flight? Raleigh can process captured imagery into products including True Orthos and 3D meshes using Esri technology before publishing the results through ArcGIS Online.
4. Why does a city digital twin need frequent updates? Construction, maintenance, development, weather events and changes to infrastructure continually alter the physical environment. A model based on outdated observations gradually becomes less useful for operational decision-making.
5. How is Raleigh using GeoAI? The city is developing models capable of identifying pavement markings from imagery. Its initial work involves arrow markings, with plans to extend detection to lane markings and crosswalks.
6. Could GeoAI replace infrastructure inspections? The material examined does not establish that. Automated image analysis can help locate or classify features and potentially identify changes, but engineering inspection and asset-management decisions require appropriate levels of verification.
7. What is the advantage of combining GIS and drone operations? Existing GIS information can support flight planning, while the resulting imagery can feed back into the same geospatial environment. This creates a repeatable cycle between existing records and new observations of physical conditions.
8. Can departments other than GIS use the system? Yes. Raleigh’s programme is relevant to parks, transportation, water, construction monitoring, inspections, emergency assessment and planning.
9. Could the system eventually detect changes automatically? Potentially, but this is not presented as an established Raleigh capability. Repeated reality capture combined with reliable feature recognition creates the technical basis for comparison, but operational change detection would require appropriate training, validation and confidence in the results.

Strategic Takeaways

1. The operational usefulness of a city digital twin depends heavily on keeping its representation of the physical environment current.
2. Integrating drone operations with existing GIS creates a repeatable information cycle rather than a collection of isolated aerial surveys.
3. GeoAI can begin converting reality-capture imagery from visual reference material into structured infrastructure information.
4. Shared capture, processing and geospatial infrastructure allows the investment to support several municipal departments rather than isolated drone programmes.
5. The progression from capturing assets to recognising them creates the possibility of eventually identifying physical changes between successive observations.