Sheffield's Digital Archive Push: Council Tackles Duplicate Image Problem in Week of Cataloguing Overhaul

Sheffield City Council confirmed this week that an internal audit of its digital asset management systems has identified more than 14,000 duplicate images stored across multiple departmental servers — a sprawl of redundant files that has been quietly inflating storage costs and slowing the work of communications and planning teams for at least three years.

The problem matters now because the council is midway through a broader digital infrastructure upgrade, budgeted at £2.1 million, that is scheduled for completion by March 2027. Project leads say that failing to resolve the duplicate image backlog before the migration goes ahead risks carrying the same inefficiencies into the new system at greater expense.

Where the Problem Runs Deepest

Two institutions are at the centre of the clean-up effort. Sheffield City Archives, based at Shoreham Street, holds digitised historical photographs dating back to the 1880s, many of which were scanned multiple times by different volunteers or contractors working under separate Heritage Lottery Fund grants. Crossover between those batches created a significant duplication problem that archivists only began formally mapping in January 2026. Separately, Sheffield Galleries and Museums Trust, which covers venues including the Weston Park Museum and the Millennium Gallery on Arundel Gate, has been running its own parallel collections database, meaning images shared between the two organisations exist in at least two incompatible formats under different file names.

The practical consequences are not trivial. Planning officers at Howden House on Union Street have, on at least two documented occasions this year, submitted planning applications to the council's own public portal with duplicate images attached — flagged only when members of the public queried inconsistencies during the consultation period for a proposed development on Abbeydale Road South.

A third strand of the problem sits with the council's communications team, which manages an external-facing media library used by local journalists and community groups. That library, which went live in its current form in September 2023, contains roughly 8,400 images. A spot-check conducted in June 2026 found that approximately one in five images had at least one near-identical duplicate under a different filename, according to figures shared at a full council briefing on 1 July.

What the Clean-Up Involves

The rationalisation programme being rolled out this week uses a combination of perceptual hashing software — technology that identifies visually identical or near-identical images even when file names differ — alongside manual review for borderline cases. The council has contracted Sheffield-based digital services firm Polestar Digital Solutions, headquartered on Eyre Street in the city centre, to run the automated phase. Manual review will be handled in-house.

The timetable is tight. The automated sweep across council servers is due to finish by 18 July, with a first report to the Digital Transformation Scrutiny Board expected on 28 July. The archives and museums trust element will follow in a second phase running through August and September, with a unified asset management protocol — covering naming conventions, metadata standards and upload permissions — set to be adopted formally before the October council recess.

The cost of the Polestar contract has not been disclosed publicly, but council documents circulated ahead of last Tuesday's briefing placed the total first-phase spend at under £40,000. That figure covers deduplication tooling, staff time and integration testing with the new server infrastructure being procured separately under the larger £2.1 million programme.

For residents and community groups who regularly download images from the council's media library — particularly neighbourhood forums in areas like Burngreave, Sharrow and the Lower Don Valley, where regeneration photography is frequently needed for grant applications — the practical advice from the communications team is to hold off on bulk downloads until after 18 July. The library will remain live, but a subset of folders covering development and planning imagery will be temporarily locked for quality review from 7 July onwards. The full library is expected to reopen with verified, deduplicated content by the end of the month.