So - unsurprisingly - in terms of software we rolled our own: https://dev.livingreviews.org/projects/epubtk/ and again unsurprisingly, being the only users of our software, the tools are nowhere near as polished as ojs. This puts us in a difficult position when trying to expand, in particular venturing into new fields of science like the humanities.
So while for quite some time our tools were descriptive - i.e. modelled after our daily work - they now became prescriptive, dictating the ways journals have to be managed.
For a new journal in democracy, we would have liked, to swap out our publication-builder component (because LaTeX as input format was considered too much of a hassle), but retain our workflow tools. Eventually the new journal decided to try with OJS, and we will have to figure out how to incorporate non-standard behaviour. Figuring out stuff like this would be a lot more fun, if it would lead to reusable software or concepts at least.
Interoperability on component level:
The resources dealt with in journal publishing are to some extent pretty obvious: The publications, metadata about publications, publication process data, some scaffolding/glue.
Another resource are user data. Generally user management should be easy to integrate with distributed authentication like OpenID or Shibboleth.
Components may be derived from the resources as functional blocks managing a single resource. So a register component may manage a journal's publication metadata; a refereeing component may encapsulate the publication workflow; a repository may be in charge of managing the publications themselves.
Again, the boundary conditions are well known. Since we deal in online publishing and the resources are well understood, an architectural approach such as REST should be the most natural way to go.
Another example for the kind of interfaces we should be interested in is OpenSearch as interface (description) to the fulltext search functionality of the repository component. Using OpenSearch made it easy for Living Reviews to swap out home-grown full text search for Yahoo! search web service.