Energy Efficiency Architecture (EEA)

Table of contents

Introduction

This proposal is part of strategy which aims to reduce global energy consumption in cloud architectures.

To achieve this goal, we follow these phases:

  1. Accurately measure the power consumption of all devices in the cloud.
  2. Analyse collected values, to place virtual machines at optimal location in the cloud.
  3. Turning off unused machines (standby, hibernation).

This document present the first of these phases.

Architecture

Here is an overview of the global layered architecture:

https://docs.google.com/drawings/d/1w9v_Ku5DTHOitt_Vu6b37nxaNq3ea7ZV3BTk6e4CRpQ/pub?w=378&h=435

And here is a more detailed figure:

https://docs.google.com/drawings/d/1rB0cS8RwaLqfsDdvnjNGU9w_IUTRcN-Y4NddojgnPA8/pub?w=918&h=1033

Here is an overview of the datacenter architecture:

https://docs.google.com/drawings/d/1Gc3ORhtluMPe3lP_uWnqQzJjX7hkXQBI23D8msrxcKU/pub?w=1315&h=700

Kwapi drivers

Driver classes (IPMI, Wattsup, etc) and superclass

Kwapi supports different kinds of wattmeters (IPMI, Wattsup, etc).
Wattmeters communicate via IP networks or serial links. Each wattmeter has one or more sensors (probes).
Wattmeters send their values quite often (each second), and they are listen by wattmeter drivers.
Wattmeter drivers are derived from a Driver superclass, itself derived from Thread. So drivers are threads.
At least one driver thread is instantiated for each wattmeter. Their constructors takes as arguments a list of probe IDs, and kwargs (specific arguments).

Driver threads roles are:

  • Setting up wattmeter.
  • Listening and decoding received data.
  • Calling a driver superclass method with measurements as argument. This method appends signature to the measurements, and publishes them on the bus.

Message format:

https://docs.google.com/drawings/d/13qIJTPp2k8w62tCrApqRNstb2vqStP_A6Ke34doKKAw/pub?w=680&h=100

Driver manager

The driver manager is the loader and the checker of driver threads.
It loads all drivers according the configuration file, and checks regularly that driver threads are alive. In case of crash, the event is logged and the driver thread is reloaded.
We can imagine that a driver will crash if a technician unplug a wattmeter, for example.

Configuration file

The file is /etc/kwapi/drivers.conf.

[DEFAULT]

# Communication
probes_endpoint = tcp://0.0.0.0:5001

# Signature
enable_signing = true
metering_secret = Change This Or Be Hacked

# Timers
check_drivers_interval = 60

# Log files
log_file = /tmp/kwapi-drivers.log

[Wattmeter 1]
probes = ['Server Taurus']
driver = Wattsup
parameters = {'device': '/dev/ttyUSB0'}

[Wattmeter 2]
probes = ['Server Orion']
driver = Dummy
parameters = {'min': 10, 'max': 20}

[Wattmeter 3]
probes = ['Desktop PC', 'Laptop']
driver = Dummy

Bus

Currently, the internal Kwapi bus is ZeroMQ. Publishers are driver threads, and subscribers are plugins.

The following design pattern is used:

ZMQ.png

Plugins can easily listen multiple data sources:

ZMQ2.png

In the near future, we plan to use OpenStack RPC framework.

Kwapi plugins

API plugin

API plugin allows Ceilometer pollster to get consumption data through a REST API.
This plugin contains a collector that computes kWh, and an API based on Flask.

Collector

The collector stores these values for each probe:

https://docs.google.com/drawings/pub?id=19VOioLPZEt0A_NAlh5RKeMMWqhVJjVOA1ExQzPYjsNo&w=905&h=100

Fields:

  • Probe id could be the hostname of the monitored machine. But it is a bit more complicated because a probe can monitor several machines (PDU).
  • Timestamp is updated when a new value is received.
  • KWh is computed by taking into account the new watt value, and the elapsed time since the previous update.
    It allows Ceilometer to compute average consumption for the duration it wants (knowing the kWh consumed and the time elapsed since its last check).
  • Watts field offers the possibility to know instantaneous consumption of a device, without having to query two times a probe in a small interval to deduce it.
    This could be especially useful if a probe has a large refresh interval: there is no need to wait its next value.

There is no history to avoid duplicating the storage architecture of Ceilometer.
The collector is cleaned periodically to prevent a deleted probe from being stored indefinitely in the collector. So when a probe has not been updated for a long time, it is deleted.

API

VerbURLParametersExpected result
 GET /v1/ - Returns detailed information about this specific version of the API.
 GET /v1/probe-ids/ - Returns all known probe IDs.
 GET /v1/probes/ - Returns all information about all known probes.
 GET /v1/probes/<probe>/ probe id Returns all information about this probe (id, timestamp, kWh, W).
 GET /v1/probes/<probe>/<meter>/ probe id, meter { timestamp, kwh, w } Returns the probe meter value.

Authentication

The pollster provides a token (X-Auth-Token).
The API plugin checks the token (Keystone request), and if the token is valid, requested data are sent.
Responses are not signed because Ceilometer trusts Kwapi plugin.

Configuration file

The file is /etc/kwapi/api.conf.

[DEFAULT]

# ACL
acl_enabled = true
acl_auth_url = http://10.0.0.2:5000/v2.0

# Signature
signature_checking = true
driver_metering_secret = Change This Or Be Hacked

# Communication
api_port = 5000
probes_endpoint = tcp://140.77.13.25:5001
probes_endpoint = tcp://140.77.10.5:10060

# Timers
cleaning_interval = 300

# Log files
log_file = /tmp/kwapi-api.log

Visualization plugin

Web interface

The visualization plugin provides a web interface with power consumption graphs. It is based on Flask and RRDtool.

VerbURLParametersExpected result
 GET /last/<period>/ period { minute, hour, day, week, month, year } Returns a webpage with a summary graph and all probe graphs.
 GET /probe/<probe>/ probe id Returns a webpage with all graphs about this probe (all periods).
 GET /graph/<period>/ period { minute, hour, day, week, month, year } Returns a summary graph about this period.
 GET /graph/<period>/<probe>/ period { minute, hour, day, week, month, year }, probe id Returns a graph about this probe.

Webpage with a summary graph and all probe graphs:

webpage.png

In the menu bar, you can choose the period for which you want to display graphs (last minutes, hour, day, week, month or year).
By clicking on a probe, you can display all graphs available for this probe, with different resolutions.

Graphs

The summary graph shows the total power consumption (sum of all the probes). Each colour corresponds to a probe.

The legend contains:

  • Minimum, maximum, average and last power consumption.
  • Energy consumed (kWh).
  • Cost.

File sizes:

  • RRD file: 10 Ko.
  • Probe graph: 12 Ko.
  • Summary graph: 24 Ko.

These is a cache mechanism: recent graphs are not rebuilt uselessly.

Configuration file

The file is /etc/kwapi/rrd.conf.

[DEFAULT]

# Communication
rrd_port = 8080
probes_endpoint = ipc:///tmp/kwapi

# Signature
signature_checking = true
driver_metering_secret = Change This Or Be Hacked

# Storage
png_dir = /tmp/kwapi-png
rrd_dir = /tmp/kwapi-rrd

# Billing
kwh_price = 0.125

# Graphs
max_watts = 200

# Log files
log_file = /tmp/kwapi-rrd.log

Ceilometer pollster

Ceilometer pollster is started periodically by Ceilometer central agent.
It knows the Kwapi URL by doing a Keystone request (endpoint-get).
It queries probe values through Kwapi API, using the GET /v1/probes/ call, so that it gets all detailed informations about all probes in just one query.
For each probe, it creates a counter object and publishes it on the Ceilometer bus.

Published counters:

  • Energy (cumulative type): represents kWh.
  • Power (gauge type): represents watts.

Counter timestamps are Kwapi timestamps, so that Ceilometer doesn't store wrong data if a probe is not updated.
Ceilometer handles correctly the case where a probe value is reset (kWh decrease), because of its cumulative type.

Conclusion

This blueprint gives an overview of what is currently implemented.

Kwapi is stored on StackForge, and is designed to be OpenStack compatible.

Compatibility in improved by using common OpenStack modules:

  • Configuration.
  • Logging.
  • Bus (if we replace ZeroMQ by OpenStack RPC).

Kwapi visualization plugin is not an essential part, but it shows how it is easy to extend Kwapi functionalities.

Your comments are welcome!


This wiki is licensed under a Creative Commons 2.0 license
XWiki Enterprise 4.5.1 - Documentation - Legal Notice

Site maintained by