Completing port of code to V2.1 API

With Juno now closed to new features we’ve started looking at what we plan to do in the Kilo development cycle. As mentioned in a previous post most of the V2 API had been implemented in V2.1 with the exception of the networking APIs. The first and highest priority will be to complete and verify the V2.1 API is equivalent to the V2 API:

• Finish porting the missing parts of the V2 API to V2.1. This is primarily networking support and a handful of patches which did not merge in the Juno cycle before feature freeze.
• Continue tempest testing of the V2.1 API using the V2 API tempest tests. Testing so far has already found some bugs and there will be some work to ensure we have adequate coverage of the V2 API.
• Encourage operators to start testing the V2.1 API so we have further verification that V2.1 is equivalent to the V2 API . It should also give us a better feeling for how much impact strong input validation will have on current users of the V2 API.

Microversions

Support for making both backwards compatible and non-backwards compatible changes using microversions is probably the second most important Nova API feature to be developed in Kilo. Microversions work by allowing a client to request a specific version of the Nova API. Each time a change is made to the Nova API that is visible to a client, the version of the API is incremented. This allows the client to detect when new API features are available, and control when they want to adapt their programs to backwards incompatible changes. By default when a client makes a request of the V2.1 it will behave as the V2 API. However if it supplies a header like:

X-OS-Compute_Version: 2.214

then the V2.1 API code will behave as the 2.214 version of the API. There will also be the ability to query the server what versions are supported. Although there was broad support for using a microversions technique the community was unable to come to a consensus on the detail of how microversions would be implemented. A high priority early in the Kilo cycle will be to get agreement on the implementation details of microversions. In addition to development work required in the Nova API to support microversions we will also need to add microversion functionality to novaclient.

API Policy Cleanup

The policy authorisation checks are currently spread between the API, compute and db layers. The deeper into the Nova internals the policy checks are made, the more work there is needed to unwind in case of authentication failure. Since the Icehouse development cycle we have been progressively been moving the policy checks from the lower layers of Nova up into the Nova API. The draft nova specification for this work is here.

The second part of the policy changes is to have a policy for each method. Currently the API gives fairly inconsistent control to operators over how resources can be accessed. Sometimes they are able to set permissions based on a per plugin basis, and at other times on an individual method granularity. Rather than add flexibility to authentication to plugins on an ad hoc basis we will be adding them to all methods on all plugins. The draft nova specification for this work is here.

Reducing technical debt in the V2/V2.1 API Code

Like many other areas of Nova, the API code has over time accumulated a reasonable amount of technical debt. The major areas we will look at addressing in Kilo are:

• Maximise sharing of unittest code
• General unittest code cleanup. Poorly structured unittests make it more difficult to add new tests as well as make it harder to debug test failures.
• API samples test infrastructure improvements. The api sample tests are currently very slow to execute and there is significant overhead when updating them due to API changes. There are also gaps in test coverage, both in terms of APIs covered and full verification of the responses for those that do have existing tests.
• Documentation generation for the Nova API is currently a very manual and error prone process. We need to automate as much of this process as possible and can use the new jsonschema input validation to help do this.

Feedback

The planning for the work that will be done in Kilo is still ongoing and the API team welcomes any feedback from users, operators and other developers. The etherpad where work items for Kilo can be proposed is here. Note that the focus of this etherpad is on infrastructure improvements to the API code, not new API features.

The Nova API team also holds meetings every Friday at 00:00UTC in the #openstack-meeting channel on freenode. Anyone interested in the future development direction of the Nova API is welcome to join.

Early in 2014 there was a very long discussion on the openstack-dev mailing list about the future of the Nova V3 API development. There were two main concerns. The first was the willingness and ability for users to port their applications from the V2 to the V3 API. The second was the the level of maintenance required to keep two Nova APIs up to date since it was becoming increasingly clear that we would not be able to deprecate the V2 API in only 2-4 cycles. As part of this discussion I wrote a document describing the problems with the V2 API and why the V3 API was developed. It also covered some ideas on how to minimise the dual maintenance overhead with supporting two REST APIs. This document describes most of the differences for clients between the V2 and V3 API.

During the Juno Design summit, the development cycle and the Nova mid cycle update there were further discussions around these ideas:

• Juno Design Summit etherpad
• Juno mid-cycle Nova meetup

Not long after the community finally reached consensus on the first part of the work required to implement a V2.1 API which is implemented using the original V3 API code. The details of the work being carried out in Juno is described in the nova specification.

In short, from a client point of view, the V2.1 API looks exactly the same as the original V2 API with the following exceptions:

• Strong input validation. In many cases the V2 API code does not verify properly the data passed to it. This can lead to clients sending data to the REST API which is silently ignored because there is a typo in the request. The V2.1 API primarily using jsonschema is very strict about the data it will accept and will reject the request if it receives bad data. Client applications will need to fixed before using the V2.1 API if they have been sending invalid data to Nova.
• No XML support. The V2 XML API is not widely used and was marked as deprecated in Juno. The V2.1 API has no support for XML, only for JSON.

From an operator’s point of view:

• The V2.1 API can be simultaneously deployed alongside the original V2 API code. By default the V2 API is exposed on /v2 and the V2.1 API on /v2.1/. This may make it easier for users to test and transition their applications over time rather than all at one time when the OpenStack software is upgraded. The V2.1 API is however not enabled by default in Juno.
• The number of extensions has been reduced. A number of extensions in the original V2 code are dummy or minimalistic extensions which were only added because adding a new extension was the only way to signal to a client that new functionality is available. In these cases the V2.1/V3 API code removed the extra extension and incorporated the newer functionality into the original extension and enabled it by default. Note that from the perspective of clients they still see the extra extensions if the functionality is enabled. So no changes are required on the client side.

Because of the late acceptance of the V2.1 specification we have not been able to merge all of the required patches to implement the V2.1 API in Juno. However, there is support for most of the equivalent of the V2 API with the exception of networking. It is expected that the remaining patches will be completed soon after Kilo opens. I will cover the V2.1 work and discussions on how we plan on handling backwards incompatible API changes in a future article.