Agility™ for Energy Reduction

Introduction

In October, 2010, Arjuna Technologies approached the management team at Newcastle University with a proposal for the University to utilise Arjuna’s Agility product in order to reduce energy consumption on IT. The project was also intended to illustrate how the University could make more effective use of its existing IT estate and manage the inevitable transition to Cloud-sourced services.

This is a summary of a report delivered to Newcastle University in June of 2011, detailing the results of that project which targeted the School of Computing Science, one of the University’s most significant consumers of IT energy. This paper also highlights the wider benefits of using Agility for additional cost savings and improved IT service within Newcastle University.

The key distinguishing feature of our approach was to deliver change through ‘enablement’ rather than ‘enforcement’. This is critical in a federated environment such as a University where many individuals and departments value their autonomy.

Tony Stevenson, Pro-Vice Chancellor at Newcastle University, said of the project "Given our experiences thus far we are hopeful of seeing an eventual saving of 48% per annum and an ROI in just over a year. The project has been delivered on-time, within budget and has caused minimal disruption to the normal working of the department."

Monitoring with Agility

A significant problem with any cost saving exercise is the ability to accurately measure outcomes and a key outcome of the project effort has been the introduction of effective and sustainable monitoring. This has been achieved through the introduction of specific monitoring policies supported by our Agility product.

Monitoring of IT inventories and activity within the University Departments was, at the time, fairly rudimentary. Departments in the University often maintain one or more partially complete, occasionally conflicting, inventories. The data collected optionally includes details of the device, its owner, location, purpose etc. Amongst other uses, the data is used to generate charts illustrating IT utilisation, in terms of the amount of time machines are on, on a per department basis. Whilst these charts are useful as broad indicators of utilisation, and therefore energy consumption, they do suffer from a number of significant problems.

First and foremost the data collected for the charts has a coarse granularity which can result in machines with utilisations of only a couple of minutes per day appearing to be on permanently. Second, the data does not reflect whether the machine is usefully working or simply sitting idle (and wasting energy). Finally, the data does not always accurately reflect changing circumstances as the data is often poorly maintained.

Within the project the quality of data concerning IT utilisation and energy consumption was significantly improved through the introduction of monitoring policies to obtain data from individual machines and from clusters. This data provided insight into how those machines were being used and was a vital precursor to any attempt at interfering with existing practices.

Power Management with Agility

The monitoring phase identified those classes of machines where the introduction of power management policy would deliver significant energy savings. One example is illustrated below.

The School supports a number of clusters for the use of students. Within the project Agility was used to enable the addition of monitoring policies for cluster machines and then, when energy consumption was better understood, additional power-saving policies were added. The results for a typical cluster are illustrated in Figure 1.

Figure 1 shows % utilisation levels (in green) i.e. user logged in, and % time on (in red) i.e. not power-saved. At the start of the project the cluster machines had no power saving whatsoever and were generally on 24x7. Power saving was introduced in two steps, the first in mid-April, the second in mid-May, and today the cluster machines are generally only fully on when being used.

The orange section illustrates a temporary period when most of the machines were powered down for reasons not associated with the project. Without the power down this orange area would have been coloured red. (The two larger spikes in utilisation in March and April were a consequence of a student using a lot of machines for an experiment but then failing to log off for some days. The monitoring has identified the need to improve auto-logout to avoid this level of waste in the future).

Deploying Agility: a framework for Policy

Agility allows the introduction of policy for groups of resources and the creation of agreements with regard to that policy. In the case of this project the policies and agreements involved are those concerned with monitoring and power management (as described above).

Figure 2 illustrates the use of Agility within the project. Deployment proceeded as follows:

1. Agility and an appropriate monitoring policy were first installed on every cluster machine. These individual machine policies were coordinated by a Cluster Manager used to initiate, control and aggregate monitoring results which could then be displayed graphically. By this means detailed energy-usage analysis, specific to individual machines and clusters could be provided.
2. Agility and a server-specific monitoring policy were installed on targeted servers. This policy discovers by whom, how much, and when, applications are utilised.
3. Agility and a PC monitoring policy were installed on a central machine from where staff machines were externally monitored. The information gathered in this case is very generic and by monitoring externally no installation was required and any disruption for staff members avoided.
4. Once monitoring had provided insight as to where Energy saving efforts should be directed, power management policy was added, again coordinated by a central policy.

At a later stage in the project we will introduce an Agility instance which will act as a ‘Departmental Manager’ used to deliver all aggregated graphical results, and ultimately provide a level of control over energy management from a central console. From the console a responsible person within the School will be able to track energy consumption and to immediately see any unexpected variance which can then be investigated, and/or dynamically modify the department’s power management policy.

Agility enables the move towards the cloud

Arjuna has demonstrated with this project that their Agility product can be used to introduce policies effectively delivering real energy savings within a short period of time.

However, whilst energy saving has been the focus of the project, Agility offers many more benefits to large organisations with federated management structures such as a University. Agility is a unique software product which can be used to improve the sharing of IT resources within an organisation, unique because it focuses on overcoming organisational barriers to sharing.

The IT industry is moving rapidly from a resource centric focus which concentrates on physical equipment and infrastructure to a services focus where lowest cost deployment of the applications software and data from the most flexible source will be mandatory. This is occurring because network connectivity has now reached a tipping point where an increasing number of applications may now be accessed easily remotely. As a consequence, all of the major application providers and hardware and software vendors are placing increasing emphasis on this new paradigm and are delivering their products as ‘cloud-based’ services. The analysts all agree, that this trend will continue to grow significantly in the coming years.

However, although the vast majority of applications and data held within many organisations are amenable to delivery from the cloud, there will (probably) always be mission-critical applications and data that must execute within the organisation’s firewall, either for reasons of security or availability. This means that organisations will increasingly be required to operate a hybrid environment with some parts of their IT maintained by their own IT department, but others (and increasingly the majority) of their IT requirements will be delivered as services from third parties. Arjuna has long argued that the ability to manage the coming transformation in the IT environment will be critical to large organisations if they wish to compete effectively.

Agility is designed to monitor and manage hybrid environments in which independent organisations, or parts of an organisation, cooperate together in order to share resources and services and by doing so reduce costs and improve service quality.

In a University’s case, services might be shared which are provided by individual departments, central IT, research partners, industry partners, government agencies, cooperating groups of Universities, or by third party vendors. Managing all of this complexity will be a key challenge over the coming decade. Success will help ensure global competiveness and will deliver service-focused, cloud-based IT which is the growing expectation of IT consumers in research, teaching, and the current generation of fee-paying students.