Case Study: Directory Services
The case study which follows describes where a Data Management Layer facilitated by Agility DataBroker has been introduced to Newcastle City Council. The example illustrates that adding a Data Management Layer need not be a disruptive process. It can be achieved incrementally and can produce immediate benefits which encourage further progress.
Newcastle City Council Directory Services
Newcastle City Council, in common with many other Local Authorities, collects information about council, commercial and voluntary service offerings from many different sources. In the case of Newcastle, and most Local Authorities, this information is delivered to citizens via a broad variety of websites, each catering to a particular demographic. In the main, each of the website providers works largely in isolation in order to collect, maintain and publish that data. As a consequence there are numerous application silos. In many cases there is considerable overlap in the demographic being served e.g. in Newcastle separate websites cater for Families, the Over 50's and for ‘harder to reach groups’. Clearly there are members of society who belong to more than one, or even all, of these groups. Consequently the same data is collected multiple times, stored in multiple systems and delivered to citizens in different formats. Not only is this wasteful of effort but it is also confusing for citizens and leads to inconsistencies. Additionally the data trapped in its silo is not available for exploitation for other purposes.
The individuals responsible for the three websites previously mentioned (‘Families Information’ – providing information for families, ‘Information Now’ – providing information for the Over 50s, and ‘InvolveNE’ – providing information for ‘harder to reach groups’) recognised the inefficiencies in the current approach and turned to Arjuna Technologies for help. Their initial situation is illustrated below. Data for all three websites (and many more not considered here) is collected independently by each team through telephone, email and paper surveys, is independently maintained and independently published on different websites. The data is not available except as published on the website.
The Agility DataBroker
Arjuna Technologies have created their Agility DataBroker product to act as a Data Management layer in just such a scenario. The project team wanted to proceed carefully with minimum disruption to the various teams, so, as a first step, Agility DataBroker was introduced to hold copies of the data collected by each independent system. This involved creating a plug-in for Agility DataBroker that was capable of obtaining the data from the existing Data Storage systems. They then created Agility DataBroker plug-ins capable of outputting the data in formats readily accessible to application developers. An interim architecture after the introduction of the Agility DataBroker is illustrated below.
Providing application developers with access to these data sets opens up the potential to deliver many new innovative applications which can benefit citizens. This opportunity becomes all the more powerful when additional data sets are added. Within this project a number of additional data sets were exposed through Agility DataBroker including a large data set provided by North Tyneside Council, a neighbour of Newcastle City. This potential has begun to be explored in an ongoing series of hackathons run by the Cloud Innovation Centre and sponsored by ENGIE.
The introduction of Agility DataBroker and the publication of the various data sets was a relatively simple process which required minimal effort and no disruption of the existing systems. However, while this first step opened up previously closed data sets, it did nothing to address the fundamental inefficiencies caused by the existence of multiple application silos. Addressing these issues requires a further series of incremental steps by which Agility DataBroker would be extended so as to deliver the data collected directly to each of the Websites. Once this has been achieved the means of collecting and storing the data can be rationalised so that duplication and inconsistencies can be removed. The eventual, intended architecture is illustrated below.
The project illustrates the problems resulting from data being consigned to application silos, and the advantages of introducing a Data Management Layer. In this case the introduction of Agility DataBroker enabled the inefficiencies in data collection and storage to be overcome, and the ability to deliver the data to multiple outputs opened up new opportunities to share the data within the organisation and beyond. Sharing the data and combining it with other data sources allows new information to be obtained with the potential to improve decision making, reducing costs and/or improving services.
Agility DataBroker was introduced without impacting upon existing applications and it should be clear that a step by step expansion to consume additional data would provide additional benefits. With data, the total exceeds the sum of the parts, as adding more data sets increases exponentially the number of ways in which data can be combined. In addition Agility DataBroker is capable of delivering transformed data in whatever formats are required. This opens up the opportunity to deliver data to both internal and external application developers in order to encourage innovation, and to citizens to improve access to Council information.
