As mentioned in the merge proposal, the API comes in two parts: infrastructure and endpoints. In 4.4, the infrastructure is now available as part of core, while the endpoints continue to only be available in the plugin.
You can think of the infrastructure as an “API construction kit”. WordPress 4.4 will make it possible for everyone to build RESTful APIs in a much easier fashion, which will benefit people building custom APIs for their site. The infrastructure handles the routing, argument handling, JSON serialisation/deserialisation, status codes, and all that other lovely REST stuff.
This is a big deal.