A model for correlation-based choreographic programming

Contributions and outline

Concretely, we provide a positive answer to our research question by focussing on Compositional Choreographies (Montesi & Yoshida, 2013)—which we build upon to show that our approach applies to both the methodology of choreographic programming and that of MST—and by presenting a formal framework relying on a model for correlation-based choreographic programming.

We call our framework Applied Choreographies. In Applied Choreographies, developers abstract from the details of correlation-based communication, and rather write high-level choreographic programs using terse and informative choreographic syntax shown in Example 1. Then, a compilation process—consisting of a set of transformations into ever-more-involved intermediate representations and a tight series of correspondences (immaterial for the programmer)—generates a correspondent set of SOC endpoints that communicate using correlation and are guaranteed to faithfully implement the behaviour specified in the source choreography. To introduce the reader to the main components of Applied Choreographies, we represent them in Fig. 1, showing their position within the framework, the relevant properties that relate them, and where we present them in this article.

Frontend Choreographies The left-most artefact (①) in Fig. 1, are Frontend Choreographies programs (Frontend Choreographies). FC is the high-level, choreographic language Applied Choreographies provides to programmers. FC provides the elements developers are used to finding in a choreographic calculus; in particular, in FC, communications happen on name synchronisation, as in standard process calculi.

Backend Calculus and FC-to-BC transformation The first branch we find in Fig. 1 departing from FC (①) is that of items ② and ④. We start from ④, depicted as an intermediate artefact in Fig. 1, which is the second calculus that we present, called Backend Choreographies (BC) (Backend Choreographies). BC has the same syntax as FC but different semantics; instead of using abstract, name-based synchronisation, BC models and keeps track of the data structures needed to implement concrete SOC, correlation-based communications (Correlation-based Communication). While more involved than FC, BC is agnostic to the specific structure and technology that define the content of the data used for correlation. The other item, ②, is a transformation procedure (FC-to-BC) that generates the data structures needed to support the execution of a source FC program using message correlation (Encoding Frontend Choreographies to Backend Choreographies and Properties). Essentially, given an FC program, we obtain a BC one which is operationally correspondent to its source FC (Theorem 1).

EPP and Endpoint Frontend Choreographies The second branch departing from FC in Fig. 1 is that of items ③ and ⑤, which regard the EndPoint Projection (EPP) transformation. The latter allows us to transform an FC program that describes the behaviour of many participants in a set of artefacts written in a fragment of FC, called endpoint Frontend Choreographies (eFC), where an eFC program describes the behaviour of a single participant. More precisely, the EPP procedure is an endomorphism (Endpoint Projection (EPP)) that transforms a source FC program into a set of eFC programs, whose syntax is restricted to only partial actions (i.e., belonging to one of the two ends of a communication). The point of the EPP step is to act as a bridge between the global actions specified by FC and the local actions of endpoint processes. The result we prove in Theorem 2 is that the transformation performed by the EPP is guaranteed to generate a set of eFC programs which, run in parallel, behave like the source FC program. Hence, the EPP allows us to consider eFC as the syntax of the intermediate artefacts in the following steps of the compilation process.

Endpoint Backend Choreographies Since FC and BC share the same syntax, the next step in the Applied Choreographies pipeline ⑥ is to assemble the EndPoint Frontend Choreographies programs from item ⑤ with the BC data structures that support correlation-based communication ④ to obtain EndPoint Backend Choreographies, which we can proceed to compile into our target local implementations.

DCC and Compilation The third calculus is the target language for the compilation, called Dynamic Correlation Calculus (DCC) ⑧ (Dynamic Correlation Calculus). DCC is a process algebra of distributed executable code, based on a low-level formal model for SOC (Montesi & Carbone, 2011). DCC models both data distribution and how concrete correlation-based communications happen. Given its low-level scope, DCC does not capture all the abstraction of choreographies. The last item from Fig. 1 is the compiler ⑦ (Compiling Frontend Choreographies into DCC Processes), which takes in the eBC programs (at step ⑥) and it synthesises a behaviour-preserving implementation as a distributed system of DCC services.

Applied Choreographies Thanks to the step-by-step transformation correspondence results of steps ②, ③, and ⑦, we build our main contribution for Applied Choreographies, which is the definition of a behaviour-preserving compiler from Frontend Choreographies to DCC distributed services—the first correctness result of an end-to-end translation from standard choreographies to programs based on a real-world communication mechanism.

Our construction lets programmers use high-level programming primitives and semantics as found in previous work on choreographies—with state-of-the-art features like asynchronous communications (Carbone & Montesi, 2013) and modular development (Montesi & Yoshida, 2013)—while our compilation procedure tackles the heavy lifting of producing correct service-oriented implementations.

We conclude our proposal by discussing related and future work in “Related Work and Discussion” and report in the Supplemental Material auxiliary technical material and the proofs of our results.

This article integrates and extends material from Giallorenzo, Montesi & Gabbrielli (2018), where we present the main ideas behind the Applied Choreographies framework. Portions of this article were previously published as part of a preprint (Giallorenzo, Montesi & Gabbrielli, 2020) and the Ph.D. thesis of one of the authors (Giallorenzo, 2016). The extensions in this work include: (a) full formal definitions (syntax and semantics of all three calculi); (b) detailed examples for each main component of the work—the three calculi and the three stages of compilation—to illustrate their relevant characteristics and features; (c) full proofs of the formal properties guaranteed by the framework (in the Supplemental Material, to avoid breaking the flow of the reader with details of the technical development). Besides the previous points, this version contains an extended, revised, and refined presentation of all the contents presented in Giallorenzo, Montesi & Gabbrielli (2018).

Applied Choreographies: an overview. Before delving into the details of our contribution, we present, through a simple example, an overview of the three languages used in this work, their runtime, and their relationship as seen through the lenses of our compilation process. To structure the overview, we take the choreography from Example 1 and consider its first two instructions. Since the description would become quite involved if we described the evolution of the program from the first instruction (the “start”), in Table 1 we assume we performed the actions that start session k (which sets the state of processes and the needed queues up to support communication) and begin by describing the status of the FC program and its related translations in BC and DCC—for brevity, we focus on processes c and s and omit to represent process b. We let c send its message and comment on the changes in the state of these systems.

Table 1:

Comparison between an FC source program and its BC encoding and DCC compilation.

DOI: 10.7717/peerj-cs.1907/table-1

In the first row called “Program” of Table 1, we find the FC program in the leftmost cell, in the central one the BC program—the same as FC—and in the rightmost one the corresponding DCC translation.

We recall that, in Applied Choreographies, FC is the only language (and semantics) exposed to the programmer, which uses it to implement the logic of a given distributed system. The other two languages, BC and DCC, are respectively a kind of intermediate representation to simplify the transition from high-level FC programs to lower-level, distributed services (in DCC) and the target language of the compilation process. Notice that below “Program” we find the “Deployment” row. Deployments introduce a remarkable difference between FC and pre-existing choreographic languages. To run an FC program we need to pair it with an FC deployment that describes the state of both its processes and session-based message queues. In the Table, we associate elements like process names and queues using pairs of the form (a,b). Specifically, we find that process c has the variable product pointing to the value ″book″, s has an “empty” state ( ∅), and both queues from C to S and S to C on session k—resp. referred by k[C⟩S] and k[S⟩C]—have no messages (the empty sequence ϵ).

Deployments allow us to transition from the name-based semantics of FC to the correlation-based one of BC without requiring the modification of the choreography. As visible from Table 1, the FC and BC programs are the same and what (considerably) changes from one model to the other is the shape of deployments. Indeed, since BC model a correlation-based communication semantics, we need to deal with more involved details: process locations (in the example, process c runs at the address clnt.com and process s at sllr.com), the data needed to support correlating messages with their intended queues, and the state of queues found at the different locations. Since the purpose of BC is to capture how communication works in SOC systems, which usually rely on XML- and JSON-formatted data (cf. “Correlation-based Communication”), we define the BC data model following a tree-like format. In Table 1, we see examples of this format for the state of processes c and s, where e.g., product_ is a leaf that points to the value ″book″. Let us focus on both branches k_ and their subtrees in the state of c and s. These structures represent the data used to communicate via correlation wrt a given session. For example, if we are process c and want to know how to find the queue where s is expecting to receive messages from us on session k, we can follow the path k.C.S_, we find ″X″ as the data structure (to keep this example simple, we use a string, but subtrees work too and the semantics of BC abstract away from this detail) that identifies/correlates with the queue. Since we assume (as in SOC) that the queue and the process that reads its messages are at the same location, we track, under the subtree k_, the location of each process; hence k.C.l_ both tracks the location of c and of all the queues it can receive messages on. The last element of BC deployments are message queues, which we identify from the combination of a location and some data. As expected, the queue at sllr.com that correlates with data ″X″—corresponding to the queue k[C⟩S] of the FC system—and the one at clnt.com that correlates with data ″Y″—corresponding to k[S⟩C] of FC—are empty ( ϵ).

Moving to DCC, the first striking difference we notice wrt FC and BC is that the status of the system (processes, queues) is not centralised into a single deployment, but it is distributed among the services that make up a network. Specifically, we find two services running in parallel ( |), resp. at the locations clnt.com and sllr.com. The services enclose two elements: a parallel composition of unnamed processes (for brevity, we omit inactive processes in the example and the corresponding composition with the parallel operator), defined by the combination of a behaviour ( Bc and Bs in Table 1) and a state ( tc and ts in Table 1). Since we already modelled a tree-shaped state for processes in BC, we adopt the same model for DCC, allowing us to take, unchanged, the state of BC processes for DCC ones. The second noticeable difference introduced by DCC is that actions are only “local”, e.g., the global FC/BC action k:c[C].product–⊳s[S].buy(x) is broken into a send ( Bc) and a reception ( Bs) actions in different processes. The syntax for communicating in DCC recalls the logic of message handling in BC. Indeed, the action in Bc sends a message on operation buy with the value of product_ to the queue in the service running at k.S.l_— sllr.com—and correlating with the data in k.C.S_— ″X″. In a complementary way, the action in Bs receives a message for operation buy, storing its payload under x_ from a queue within its enclosing service that correlates with k.C.S_—again, ″X″.

Closing our overview, we look at the bottom pair of rows in Table 1, after we let c (and its corresponding DCC process) send its message. For brevity, we report in these rows only the elements changed from the previous ones. At the level of choreographies (FC and BC), we reduced the program so that the next instruction we might execute is the reception by s of the received message. This kind of unfolding underlines how FC and BC model asynchronous communication, i.e., at runtime the global communication action breaks into the delivery of a message to a queue and its residual receive action in the program. Looking at the deployments of both FC and BC, we find the message sent by c in the corresponding queue for s. Similarly, in DCC we let the process at clnt.com send its message, so that Bc reduces to 0 (inaction). As expected, we find the message in the queue correlating with ″X″ at sllr.com.

Syntax of frontend choreographies

We present the formal syntax of FC, shown in Fig. 3. In the remainder, we use the symbol ∼ over an element as an ordered set of that kind of elements, e.g., p~ indicates an ordered set of processes p1,…,pn.

Complete Actions In term (start), process p creates a new session k together with processes q~ ( q~ is assumed non-empty). Process p, called active process, is already running, whereas each process q in l.q~, called service process, is dynamically created at the respective service location l. Each process is annotated with the role it plays in the new session k. Term (com) reads: on session k, process p sends to process q a message for its operation o; the message carries the evaluation of expression e on the local state of p, whilst x is the variable where q will store the content of the message. We leave the guest language for writing local expressions ( e) unspecified, and assume that it consists of terms for accessing local variables ( x) and implementing standard computations based on those (e.g., arithmetics).

Partial Actions A choreography can use partial actions to interact with other choreographies composed in parallel. Thus, Partial actions describe the behaviour of processes that wish to synchronise with “external” participants. Concretely, these external participants will be processes and/or services whose behaviour is defined in other choreographies composed in parallel. In (req), process p requests some external services, respectively located at l~, to create a new session k and some new external processes. Role annotations follow the same intuition as in term (start): in the new session k, p will play A and each new external process qi will play the respective role Bi. Term (acc) is the dual of (req) and defines a choreography module that provides the implementation of some service processes. In term (send), process p sends a message to an external process that plays B in session k. In term (recv), process q receives a message for one of the operations oi from an external process playing role A in session k, and then proceeds with the corresponding continuation. In the remainder, we omit curly brackets in (recv) when they have only one operation, i.e., k:A–⊳q[B].o(x);C is an abbreviation of k:A–⊳q[B].{o(x);C}.

Other Terms Term (seq) is sequential composition. In a conditional (cond), process p evaluates a condition e in its local state to choose between the continuations C1 and C2. Term (par) is standard parallel composition, which allows partial actions in two choreographies C1 and C2 to interact. Respectively, terms (def), (call), and (inact) model the definition of recursive procedures, procedure calls, and inaction. Some terms bind identifiers in continuations—the choreography that follows them in a sequential composition. In terms (start) and (acc), the session identifier k and the process identifiers q~ are bound (as they are freshly created). In terms (com) and (recv), the variables used by the receiver to store the message are bound ( x and all the xi, respectively). In term (req), the session identifier k is bound. Finally, in term (def), the procedure identifier X is bound. In the remainder, we omit 0 or irrelevant variables (e.g., in communications with empty messages). Terms (com), (send), and (recv) include role annotations only for clarity reasons; roles in such terms can be inferred, as shown in Montesi (2013).

Semantics of frontend choreographies

We give an operational semantics for FC in terms of reductions of the form D,C→D′,C′, where D is a deployment. In FC, deployments keep track of the local states of processes (the values of their local variables) and the messages in transit in sessions, which we use to model asynchronous communications. While similar to other concepts, such as configurations and state, we name the runtime companions of choreographies “deployments” because they represent the environment where the software (the choreography) is deployed and executed, and we understand the term as more general than configurations/state—e.g., FC deployments contain both the state of processes and the configuration and state of queues. In the following, we formalise our notion of deployment for FC and we present its reduction semantics.

Frontend deployment transitions

In our semantics, choreographic actions have effects on the state of a system—deployments change during execution. At the same time, a deployment also determines which choreographic actions can be performed. For example, a communication from role A to role B over session k requires a queue k[A⟩B] to exist in the deployment of the system. We formalise the notion of which choreographic actions are allowed by a deployment and their effects using transitions of the form D,δ▸D′, read “the deployment D allows for the execution of δ and becomes D′ as result”. The following grammar defines δ actions.

δ::=startk:p[A]⊲–⊳l.q[B]~(sessionstart)|k:p[A].e–⊳B.o(sendinsession)|k:A–⊳q[B].o(x)(receiveinsession)

The rules defining D,δ▸D′ are given in Fig. 6.

Rule ⌊D|Start⌉ creates a new session k between an existing process p and new processes q~ by updating the deployment with: a new (empty) state for each of the new processes q in q~ ( [q↦∅∣q∈q~]); and a new (empty) queue between each pair of distinct roles in the session ( [k[C⟩E]↦ϵ∣{C,E}⊆{A,B~}]).

Rule ⌊D|Send⌉ models the effect of a send action. In the first premise, we use the auxiliary function eval to evaluate the local expression e in the state of process p, obtaining the value v to use as message payload. Then, in the conclusion, we add a message (o,v)—where o is the operation used to label the message—to the tail of the queue k[A⟩B], i.e., the queue expected to contain messages sent by A to B in session k. We assume that function eval always terminates—in practice, this can be obtained by using timeouts.

Rule ⌊D|Recv⌉ models the effect of a reception. In the premise, we get the head of the message queue between the sender and receiver, i.e., (o,v), which we remove in the conclusion from the queue ( [k[A⟩B]↦m~]), updating the variable used to store the message in the state of the receiver ( [q↦D(q)[x↦v]]).

Reductions

We define the rules for reductions D,C→D′,C′ using deployment transitions. We call D,C a running choreography. The reduction → for FC is the smallest relation closed under the rules given in Fig. 7.

Rule ⌊C|Start⌉ creates a new session making sure that both the new session name k′ and processes r~ are fresh wrt D ( D#k′,r~). We use the fresh names in the continuation C (via standard substitution C[k′/k][r~/q~]).

Rule ⌊C|Send⌉ reduces a send action, if the deployment permits it: D,k:p[A].e–⊳B.o▸D′. Rule ⌊C|Recv⌉ reduces a message reception, if the deployment permits the reception of a message on one of the branches in the receive term ( j∈I). Recalling the corresponding rule ⌊D|Recv⌉, this can happen only if the deployment D has a message for operation oj in the queue k[A⟩B].

Rule ⌊C|Eq⌉ closes → under the congruences ≡C and ≃C. Structural congruence ≡C, reported in Fig. 8, is the smallest congruence supporting α-conversion, recursion unfolding, and commutativity and associativity of parallel composition. The swap relation ≃C, reported in Fig. 9, is the smallest congruence able to exchange the order of non-interfering concurrent actions. For example, provided pn returns the set of process names, Rule ⌊CS|EtaEta⌉ swaps two communications respectively enacted by completely disjoint processes. Rule ⌊C|Eq⌉ also enables the reduction of complete communications on (com) terms—see the last equivalence in Fig. 8, which unfolds a complete communication term into the two corresponding send and receive terms. Rule ⌊C|PStart⌉ starts a new session by synchronising a partial choreography that requests to start a session with other choreographies that can accept the request. The premise of the rule {l.B~}=⊎i⁡{li.Bi~}i, where ⊎ indicates the disjoint union of the list of located roles, requires that in the accepting choreographies the list of locations and their supported roles match the corresponding list of the request. The rest of the rule is similar to ⌊C|Start⌉. Conveniently, deployment transitions allow us to syntactically equate the effect of starting a session with either a complete start or the partial composition of partial actions. The choreographies accepting the request remain available for subsequent reuses.

Rules ⌊C|Cond⌉, ⌊C|Ctx⌉, and ⌊C|Par⌉ respectively model guarded conditionals, recursion, and parallel composition in a standard way.

Example 3. The interplay between ≃C and rule ⌊C|Send⌉ yields an elegant formalisation of asynchronous behaviour for choreographies that, contrary to previous work (Carbone & Montesi, 2013), does not require a labelled transition system and ad-hoc rules. Consider Line 10 in Example 2, reported below.

C=def⁡k:b[B]–⊳c[C].ok();k:b[B]–⊳s[S].ok()

We can reduce C as follows (for brevity, we omit deployments):

C→k:B–⊳c[C].ok();k:b[B]–⊳s[S].ok()by⌊C|Eq⌉ and ⌊C|Send⌉→k:B–⊳s[S].ok();k:B–⊳c[C].ok()by ⌊C|Eq⌉ and ⌊C|Send⌉

In this case, process s may receive its message before process c, due to asynchronous message passing (the sending actions for process b are non-blocking).

Correlation-based communication

Processes in SOC run within services and communicate asynchronously. To realise asynchronous communication, services provide an unbounded number of first-in-first-out message queues that processes interact with. The interaction happens from processes that associate a message insertion/retrieval action with a correlation key, which uniquely identifies the queue subject of the action. Concretely, a correlation key corresponds to a set of data that the service associates to a specific queue.

Processes retrieve messages from the queues of their enclosing service, as represented in (the right side of) Fig. 10 by process r1, which wants to consume a message received on queue Q1, associated to the correlation key k1. The request is satisfied by the service, which delivers message m1 to r1, also removing the interested message from the head of queue Q1. The complement of the action above is message insertion. Any process (within the queue-enclosing service and remote) can insert data into a queue by sending a message to the service owning the queue. That message must associate the payload with the correlation key that identifies the queue within the service. Concretely, when a service receives a message from the network, it inspects its content, looking for a valid correlation key, i.e., one that points to any of its queues. If a queue can be found, the message is enqueued in its tail. In Fig. 10, this is represented by data k1 marked by the attribute key in the message sent by process pn (of Service1) to Service2. At reception, Service2: (1) checks for the presence of the attribute key; (2) extracts the corresponding key k1; (3) finds the queue Q1, pointed by k1; (4) enqueues the received payload in Q1 as message mn.

As depicted in Fig. 10, messages in SOC contain correlation keys as either part of their payload or in some separate header. As in Montesi & Carbone (2011), also here we abstract away such details. To summarise, two processes can communicate over correlation-based messaging if: (i) the sender knows the (location of the) service where the addressee is running and (ii) the sender and the addressee know the key corresponding to a queue in the addressee’s service. After having presented the mechanism of correlation for message exchange, we can proceed to explain how we model SOC systems in BC.

Data and Process State. Data in SOC is structured following a tree-like format, e.g., XML (Bray et al., 1998) or JSON (Bray, 2017). In BC, we use trees to represent both the payload of messages and the state of running processes (as in, e.g., BPEL (OASIS, 2007) and Jolie (Montesi, Guidi & Zavattaro, 2014)).

Formally, we consider rooted trees t∈T, where T=Val∪L∪Set(Lab×T) and

t::=v|l|{x1―:t1,…,xn―:tn}

i.e., a tree (node) is either a value v, a location l, or a set of ordered pairs of edge labels x_,y_∈Lab and tree nodes. We assume tree nodes to be values or locations only in leaves. Given this definition of trees, we define BC variables as paths on trees (the latter, we remind, represent states of processes) as sequences of labels x,y∈Seq(Lab) such that x::=x_.x|ϵ, where ϵ is the empty sequence, which we often omit for brevity. When writing paths in their extended form, e.g., x_.y_.z_.ϵ, we often use the abbreviation x.y.z_.

In addition, we define two operators to handle trees: path application and deep copy. The path-application operator x(t) is used to access the sub-nodes pointed by path x in tree t. Intuitively, x(t) returns either the value, the location or the sub-tree pointed by path x in t. If x is not present in t, x(t) returns an empty set of ordered pairs label-tree. Formally,

x_.x(t)={x(x_.ϵ(t))ifx≠ϵt′ifx=ϵandt={x_:t′,…,xn_:tn}∅otherwise

The deep-copy operator t◃(x,t′) is a (total) replacement operator that returns the tree obtained by replacing in t the sub-tree rooted in x(t) with t′. If x is not present in t, t◃(x,t′) adds the smallest chain of empty nodes to t such that it stores t′ under path x. Formally,

t◃(x_.x,t′)={∅◃(x_.x,t′)ift∈Val∪L(t\{x_:x_(t)})∪{x_:t′}ift∉Val∪Landx=ϵ(t\{x_:x_(t)})∪{x_:x_(t)◃(x,t′)}otherwise

Backend deployments, transition rules, and BC semantics

On top of the convention of using the terms “Frontend choreography/deployment” to indicate a Frontend Choreographies program/deployment, in the remainder we adopt the same convention for “Backend choreographies” and “Backend deployments”. We use the term “choreography” alone, when the context makes it clear when we refer to Backend or Frontend choreographies. We define the notion of deployment for BC, denoted ID, which includes: (1) the locality of processes; (2) queues, pointed by a combination of a location and a correlation key; (3) the state of processes. Formally, ID is an overloaded partial function defined by cases as the sum of three partial functions gl:L⇀Set(P), gm:(L×T)⇀Seq(O×T), and gs:P⇀T. The domains and co-domains of the functions are disjoint, hence:

ID(z)={gl(z)ifz∈L,gm(z)ifz∈(L×T),gs(z)otherwise

Function gl maps a location to the set of processes running in the service at that location. Given a location l, we read ID(l)={p1,…,pn} as “the processes p1,…,pn are running at the location l”, assuming that each process p runs at most at one location. Function gm maps a pair location-tree to a message queue. This reflects message correlation as informally described above, where a queue resides in a service, i.e., at its location and is pointed by a correlation key. Given a pair l:t, we read ID(l:t)=m~ as “the queue m~ resides in a service at location l and is pointed by correlation key t”. The queue m~ is a sequence of messages m~::=m1::⋯::mn|ϵ and a message of the queue is m::=(o,t), where t is the payload of the message and o is the operation on which the message was received. Pairing operation labels with message payloads is typical of SOC implementations in general (as it happens, e.g., in SOAP messages (Mitra & Lafon, 2003)). Indeed, while not essential for the correct delivery of messages, operation labels are used by processes to program external choices (for instance, a process expecting to receive a message on either of two mutually-exclusive operations, e.g., to continue or exit a loop). The case applies also to BC, where we preserve the association between payload and operations (o,t)—similarly to FC (o,v). Function gs maps a process to its local state. Given a process p, the notation ID(p)=t means that p has local state t.

Backend Deployment Transitions In BC, we replace the deployment transitions of FC (commented in “Semantics of Frontend Choreographies”, rules in Fig. 7) with the rules for ID,δ▸ID′, reported in Fig. 11, explained below.

Rule ⌊ID|Start⌉ simply retrieves the location of process p (the one that requested the creation of session k) and uses rule ⌊ID|Sup⌉ to obtain the new deployment ID′ that supports interactions over session k. Namely, ID′ is an updated version of ID with: (i) the newly created processes for session k and (ii) the queues used by the new processes and p to communicate over session k. In addition, in ID′, (iii) the new processes and p contain in their states a structure, rooted in k_ and called session descriptor, which includes all the information (correlation keys and the locations of all involved processes) to support correlation-based communication in session k. Formally, this is done by rule ⌊ID|Sup⌉ where we ① retrieve the starter process, here called q1, which is the only process already present in ID. Then, given a tree t, we ensure it is a proper session descriptor for session k, i.e., that: ② t contains the location li of each process, represented by its role in the session Bi, under path Bi.l_; ③ t contains a correlation key tij for each ordered couple of roles Bi, Bj under path Bi.Bj_, such that ④ there is no queue in ID at location lj pointed by correlation key tij. Finally, we assemble the update of ID in four steps: ⑤ first, we obtain ID′ by adding in ID the processes q2,…,qn at their respective locations; ⑥ second, we obtain ID″ by adding to ID′ an empty queue ϵ for each pair lj:tij; ⑦ third, we obtain ID′′′ from ID′′ by storing in the state of (the starter) process q1 the session descriptor t under path k_; ⑧ and we update ID′′′ such that each new created process ( q2,…,qn) has in its state just the session descriptor t rooted under path k_. We deliberately define in ⌊ID|Sup⌉ the session descriptor t with a set of constraints on data, rather than with a procedure to obtain the data for correlation. In this way, our model is general enough to capture different methodologies for creating correlation keys (e.g., UUIDs or API keys). Rule ⌊ID|Send⌉ models the sending of a message. We comment on the premises. From left to right, the first gets the location l of the receiver B from the state of the sender p; the second retrieves the correlation key in the state of p (playing role A) to send messages to role B; the third evaluates the expression e of the sender p using its local state to get a value tm. The function eval evaluates expressions in a process state, traversing its paths and performing local computation. We highlight that, since in BC we preserve the syntax of Fronted Choreographies, we make two assumptions: that expressions (e.g., e in ⌊ID|Send⌉) are defined on Variables and that eval in BC automatically maps variables x, y, z into the respective paths x_.ϵ, y_.ϵ, and z_.ϵ, used to access the process states in ID. Finally, in the conclusion of the rule, we add the message (o,tm) in the queue pointed by l:tc that we found via correlation.

Rule ⌊ID|Recv⌉ models a reception. From left to right, the first premise finds the correlation key tc for the queue that q (playing role B) should use to receive from A in session k. The second premise retrieves the location l of q. The third accesses the queue pointed by l:tc and retrieves message (o,tm). The last premise updates ID to ID′ removing (o,tm) from the interested queue. Dually to rule ⌊ID|Send⌉, where eval maps variables into paths, in the conclusion of rule ⌊ID|Recv⌉ we map x, i.e., the intended variable that should store the payload tm in the state of q, into path x_.ϵ.

Encoding frontend choreographies to backend choreographies and properties

Now that we have presented Backend Choreographies, we can proceed with defining a compilation procedure from high-level FC programs to low-level services. Here, we tackle the transition from FC programs to their intermediate representation toward SOC systems as Backend Choreographies. Specifically, we translate FC programs that use the abstract mechanism of communication over names, into BC programs that use the concrete mechanism of correlation-based communication. We prove our translation correct, i.e., that our encoding guarantees an operational correspondence between the semantics of a Frontend choreography and its Backend encoding. Formally, since choreographies in BC have the same syntax as FC ones, we can translate FC runtime terms D,C to BC runtime terms by encoding the FC deployment D to an appropriate Backend deployment. Below, we define the encoding in the form of an algorithm for clarity and compactness. Notably, BC deployments contain more information wrt FC deployments. We extract this data from Γ, the typing environment of D,C.

What the algorithm ⟪D⟫Γ does is: 1. include in ID all (located) processes present in D (and typed in Γ); 2. translate the state (i.e., the association Variable- Value) of each process in D to its correspondent tree-shaped state in ID; 3. for each ongoing session in D, set the proper correlation keys and queues in ID and, for each queue, import and translate its related messages.

More precisely, in the algorithm defined in Fig. 12 at Line 1, we create a new Backend deployment ID and assign to it the totally undefined function ( ∅); ID is an empty Backend deployment. Then, following Lines 2–13, we make the following updates on ID: Lines 2–4, for each located process p@l in Γ, we update the locations of ID to contain p at location l (Line 4) and we include process p in ID, associating to it an empty state, i.e., the empty tree ∅ (Line 4); Lines 5–6, for each variable x (typed in Γ) of a process p, we update the state of process p in ID to include the association of x to its value in the state D(p). As done in rules ⌊ID|Send⌉ and ⌊ID|Recv⌉, we map FC variables x∈Var into BC paths x_∈Seq(Lab); Lines 7–13, follow the same principles to support correlation-based exchanges as formalised in rule ⌊ID|Sup⌉; for each couple of processes p,q, respectively playing distinct roles A and B in a session k, with q located at l. Lines 8 , we obtain a fresh correlation key t with auxiliary function fresh. The latter takes deployment ID and location l as input and returns a correlation key which is fresh among the keys associated to location l in ID. Formally t is such that l:t∉dom(ID); Lines 9, we associate correlation key t with location l in ID and make it point to the corresponding queue of messages from role A to role B in D (accessed with triple k[A⟩B]). Note that we can directly copy message queues from D into ID. Indeed, while message queues in D and ID are respectively of type Seq(O×Val) and Seq(O×T), by definition T subsumes Val; Lines 10–11, we include in the state of processes p (Line 10) and q (Line 11) correlation key t, storing it under path k.A.B_; Lines 12–13, we include in the state of processes p (Line 12) and q (Line 13) the location of role B under path k.B.l_.

The encoding from FC to BC guarantees a strong operational correspondence.

Note that, since we use Γ to carry the information needed to synthesise a BC deployment from an FC one, such that Γ⊢D,C, we are restricting the set of FC programs that we encode to BC to only the well-typed ones. For example, in well-typed Frontend choreographies, no process can be present in more than one choreography composed in parallel and, thus, there is inter-process but no intra-process parallelism. In general, we would not need to impose one such restriction for supporting the encoding of FC deployments to BC ones. However, in the later steps of our compilation pipeline, we restrict the class of compilable programs to well-typed ones. Thus, we prefer to avoid introducing a dedicated environment for this step—which would further involve our development—and we rather adopt the minimal solution of supporting the encoding via the typing environment.

We report in Section 3.3 of the Supplemental Material the proof of Theorem 1. Intuitively, we can prove (Completeness) by induction on the derivation of D,C. The main observation is that the encoded system ⟪D⟫Γ,C mimics D,C by applying the same semantic rules on C and the corresponding deployment transitions (e.g., respectively defined by rules ⌊D|Send⌉ and ⌊ID|Send⌉). Let ID′ be the Backend environment obtained from the reduction ⟪D⟫Γ,C→ID,C′ on rule ⌊C|Start⌉. Since the encoding algorithm ⟪D⟫Γ (cf. Fig. 12) and the rule ⌊ID|Sup⌉ (on which rule ⌊ID|Start⌉ relies) implement the same principles, we know that k.A.B_(ID) and k.A.B_(⟪D′⟫Γ′) will be the same, except possibly for (i) the location of processes and (ii) trees of correlation keys corresponding to the same paths. Concretely, item (i) derives from the fact that Γ and Γ′ can disagree on the location of the same process p, and item (ii) is caused by the random generation of correlation keys, for which, considering a correlation key rooted in k.A.B_ of a process p, the trees obtained from k.A.B_(ID(p)) and k.A.B_(⟪D′⟫Γ′(p)) may differ. However, these discrepancies do not constitute a problem, since both locations and correlation keys are used consistently in their respective deployments, which are thus interchangeable. This observation allows us to surmise that, without loss of generality, we can consider the case that Γ and Γ′ agree on the location of the services and that the random generation of the correlation keys coincides, making the equation hold.

The same holds for (Soundness), which we can prove by induction on the derivation of ⟪D⟫Γ,C.

Endpoint projection (EPP)

Given a choreography³ C, its Endpoint Projection (EPP), denoted ⟦C⟧, returns an operationally-equivalent composition of Endpoint choreographies. Intuitively, an Endpoint choreography is a choreography that does not contain complete actions—i.e., terms (start) and (com)—and that describes the behaviour of a single process. We also recall that a choreography can contain two kinds of processes: active processes which are already running, and service processes which accept requests to create new active processes at their respective associated location l. As detailed later on, our EPP procedure projects Endpoint choreographies onto all processes, both active and service ones.

Our definition of EPP is an adaptation of the one presented in Montesi & Yoshida (2013) and it is divided into two components: (1) a process projection that derives the Endpoint choreography of a single process p from a given choreography C, denoted ⟦C⟧p; (2) the actual EPP of a given choreography C, which results in the parallel composition of: (2a) the process projections of all active processes in C; (2b) the process projections of all service processes in C, with the exception that we merge into the same Endpoint choreography all projections of service processes that accept requests at the same location.

We first present the process projection and then the actual Endpoint Projection.

Process Projection We define process projection starting from formalising Endpoint choreographies.

The process projection of a subject process p in a choreography C, returns the Endpoint choreography obtained following the rules defined in Fig. 15. Process projection follows the structure of the source choreography. We briefly comment the rules in Fig. 15, from top to bottom. We start with the complete actions (start) and (com) which, if the subject process takes part in them, are projected onto proper partial terms. When projecting a (start) action, if the subject process is the active process p, we project a (req). Otherwise, if the subject process is one of the service processes in q~, we project an (always-available) (acc). Similarly, when projecting a (com) action, if the subject process is the sender or the receiver in the interaction, we respectively project a (send) or a (recv). Partial actions (acc), (req), and (send) are projected verbatim, except for (acc) terms, which define the availability of only the subject process. When projecting a (rec) term, we project both the body of the procedure ( C′) and the choreography C. This is safe even if r does not take part in the body of X; indeed, in that case, the projection of C′ is just an (inact) term. As a consequence, we can safely project (call) terms verbatim. The projections of conditionals and receptions are peculiar: we project a conditional verbatim if the subject process evaluates the condition; for all other processes, we merge their behaviour with the merging (partial commutative) operator ⊔, defined by the rules reported in Figure 8 of the Supplemental Material. We define C⊔C′ only for Endpoint choreographies, returning a choreography isomorphic to C and C′ up to receptions including all receptions with distinct operations. We use ⊔ also in the projection of (recv), where we require the merging of the behaviour of all processes not receiving the message. The projection of two choreographies in parallel is the parallel composition of their projections and (inact) is projected verbatim.

Note the definition of the rule of process projection for (rec) terms. Indeed, applying a naïve rule like

⟦defX=C′inC⟧r=defX=⟦C′⟧rin⟦C⟧rin the EPP would yield more than one procedure with the same identifier, which could prevent the obtained projection from being typable as, according to the typing rules defined in “Typing”, we cannot have in Γ two definition typings on the same identifier. To tackle the issue, the rule for (rec) terms in Fig. 15 guarantees the coherent definition and usage of process-unique identifiers through renaming. The renaming is safe as, by assumption, we consider well-sorted choreographies where definitions always precede recursive calls. We conclude this paragraph with the formal definition of process projection.

Endpoint Projection We proceed to define our Endpoint Projection. In the definition below, we use the grouping operator ⌊C⌋l, which returns the set of all service processes accepting requests at location l. We report in Figure 9 of the Supplemental Material the rules that inductively define ⌊C⌋l.

Definition 5 states that the EPP of a choreography is the parallel composition of two kinds of Endpoint choreographies: (i) Endpoint choreographies that are the process projection of active processes p∈fp(C) and (ii) Endpoint choreographies that are the merge ( ⊔) of the process projections of all service processes available at the same location l, i.e., p∈⌊C⌋l.