Partage dinformation à grande échelle Marc Shapiro Cambridge Distributed Systems Group

1. Résumé des épisodes précédents…

Soutenance HDR Gestion répartie d'objets 2 Grandes étapes 1978: Ingénieur ENSEEIHT : doctorant LAAS : Post-doc MIT Lab. for Computer Science : Chargé de Recherches CMIRH : INRIA : Directeur de recherche, responsable scientifique INRIA projet SOR : Cornell 1997: Jini, Sun Research Labs 1999+: Senior Researcher, MSR Cambridge

Soutenance HDR Gestion répartie d'objets 3 Thèses encadrées Yek Loong Chong. U. de Cambridge, Nicolas Richer. Paris 6, Fabrice le Fessant (co-encadrement). Polytechnique, Xavier Blondel. CNAM, Aline Baggio. Paris 6, Georges Brun-Cottan. Paris 6, Julien Maisonneuve, Paris 6, Paulo Ferreira. Paris 6, Hervé Soulard. Paris 6, David Plainfossé. Paris 6, Daniel Edelson. UC Santa Cruz, Michel Ruffin. Paris 6, Yvon Gourhant. Paris 6, Sabine Habert. Paris 6, Mesaac Mounchili Makpangou. Paris 6, 1989.

Soutenance HDR Gestion répartie d'objets 4 Publications Computing Surveys, en cours PODC 2001 Livre Springer 2000 PLDI 1998 ECOOP 1998 ICDCS 1996 IWMM 1995 WDAG 1995 Livre IEEE 1994 OSDI 1994 ICDCS 1994 PODC 1992 RDS 1991 Computing Systems 1989 SOSP 1989 ICDCS 1986

Soutenance HDR Gestion répartie d'objets 5 Le partage de linformation à grande échelle Complexité des systèmes répartis : parallélisme, pannes, événement, latence Objets: sémantique non prédéfinie Système : outils communs aspects dynamiques compromis Traiter les problèmes de fond, impact sur le long terme

Soutenance HDR Gestion répartie d'objets 6 Objets Fragmentés Structuration Contrôle de la transparence Mandataire programmable spécifique au client état du protocole Système SOS Voir proxy Web Jini pages Web dynamiques

Soutenance HDR Gestion répartie d'objets 7 Chaînes de Paires Souche- Scion Automatiser la gestion dobjets en réparti Références RM locaux + coordination asynchrone Simplicité + efficacité Tolérant les pannes détectables Dérivation formelle x y z t

Soutenance HDR Gestion répartie d'objets 8 RM asynchrone en mémoire répliquée Scalability asynch. Consistency Local GC Distributed GC Sufficient safety rules Union rule Clean propagation Comprehensive scanning Create before delete Causal delivery PerDiS

Soutenance HDR Gestion répartie d'objets 9 Réplication optimiste Partage en écriture décentralisé IceCube : moteur général de réconciliation journalisation dopérations paramétré par la sémantique Cohérence à terme : modèle : opérations + contraintes sûreté + vivacité invariant global décrire et comprendre les solutions Très préliminaire

2. Réplication optimiste et IceCube

Soutenance HDR Gestion répartie d'objets 11 Optimistic replication Replicas of shared objects on sites Without synchronisation: peer-to-peer read and update! Applications: high latency networks disconnected operation cooperative work Improves availability & performance Consistency: a posteriori, offline Merge independent updates

Soutenance HDR Gestion répartie d'objets 12 Example: cooperative engineering with CVS CVS: developing shared code Local, disconnected replica: no interference Conflicts: Write same file = syntactic Overlap in file = violates edit semantics Doesnt compile, test = violates application semantics Both sides of a conflict are excluded Manual repair

Soutenance HDR Gestion répartie d'objets 13 Example: Bayou General-purpose database Any replica can update, log actions action = { dependency check, operation, merge-procedure } Optimistic replication: epidemic exchange logs { roll-back, replay }*; commit dep-check: semantic check for conflict merge-proc: semantic repair

Soutenance HDR Gestion répartie d'objets 14 Operation-based model scheduling commitment

Soutenance HDR Gestion répartie d'objets 15 Execution model operation = code + pre/post-conditions Schedule must satisfy Violation conflict But pre/post-conditions often unknown Conservative approximations pre(x 0 )post(x 0, f(x 0 )) x 1 := f(x 0 ) pre(x 0 )post(x 1, g(x 0 )) x 1 := g(x 0 ) pre(x 1 )post(x 1, g(x 1 )) x 2 := g(x 1 )

Soutenance HDR Gestion répartie d'objets 16 Happens-before True constraints unknown e 1 precedes e 2 in process e 1 sends, e 2 receives e 1 e 2 (e 1 e 2 ) (e 2 e 1 ) e 1 || e 2 e 1 || e 2 : e 1 does not cause e 2 e 1 e 2 : e 1 might cause e 2 Partial order, consistent with causal dependence Schedule consistent with

Soutenance HDR Gestion répartie d'objets 17 Syntactic vs. semantic mechanisms Scalar timestamps no concurrency detection very conservative approx. of causality Vector timestamps detect concurrency conservative approx. of causality Alternative: explicit constraints

Soutenance HDR Gestion répartie d'objets 18 Constraints between operations Not all schedules are acceptable Constraints, e.g. x > 50 respect causal ordering all-or-nothing transactions alternative execution paths conflicting operations exclude each other

Soutenance HDR Gestion répartie d'objets 19 IceCube: Primitive constraints Constraint = predicate (action, schedule) Declarative (static binary): MustHave: a b if a s and a b then b s (not necessarily contiguous nor in order) Order: a b if a, b s and a b then a before b in s (not necessarily both nor contiguous) Imperative (dynamic): a.preCondition (State)

Soutenance HDR Gestion répartie d'objets 20 IceCube: log constraints parcel predecessor- successor alternatives Express user intents: Predecessor/successor: a b b a b uses effect of a; a causes b Parcel: a b b a transaction Alternatives: a b b a

Soutenance HDR Gestion répartie d'objets 21 IceCube: Object constraints Shared data type advertises static semantics mutually exclusive a b b a best order (e.g. bank: credits before debits) a b Only between concurrent actions Also: dynamic constraints commute best order mutually exclusive

Soutenance HDR Gestion répartie d'objets 22 Optimistic concurrency control & scheduling Two actions are either: Dependent schedule in dependence order Commutative schedule in any order Concurrent with favourable order schedule in non-conflicting order Concurrent and conflicting or exclude one, the other, or both

Soutenance HDR Gestion répartie d'objets 23 IceCube scheduling Insight: conflict: choice of which action to exclude maximise value

Soutenance HDR Gestion répartie d'objets 24 IceCube scheduling model log constraints object constraints dynamic constraints

Soutenance HDR Gestion répartie d'objets 25 Search vs. syntactic order

Soutenance HDR Gestion répartie d'objets 26 Performance of IceCube heuristics

3. Cohérence à terme (Eventual consistency)

Soutenance HDR Gestion répartie d'objets 28 Eventual consistency Consistent with user intents Consistent with data invariants Replicas consistent with each other Eventual consistency: Each site receives all actions Schedule that satisfies constraints Common stable prefix Equivalent results

Soutenance HDR Gestion répartie d'objets 29 Stability Peer-to-peer, indefinite tentative update + advisory reconciliation OK But stability needed: Users, external world depend on it Garbage collect multilog Only stable actions relevant for consistency Stable: eventually decisions not changed Committed: definitely included in all schedules Aborted: definitely excluded

Soutenance HDR Gestion répartie d'objets 30 Eventual consistency: intuitive Liveness: sites receive all operations Epidemic multicast Quickly Safety: sites compute the same value Equivalent schedules Stability: actions eventually not undone Commit / abort

Soutenance HDR Gestion répartie d'objets 31 Sound schedules s sound = s satisfies constraints for all its actions: Closed for MustHave a s a b b s Consistent with Order ( acyclic) (a,b s a b) a before b in s Actions succeed a s a.preCondition (state)

Soutenance HDR Gestion répartie d'objets 32 Maintaining local soundness site i, schedule s i : Legal: committed i s i aborted i s i = Safe: s i sound When aborting a, also abort actions that MustHave a When committing a, also abort uncommitted actions that are Ordered before a

Soutenance HDR Gestion répartie d'objets 33 Schedule equivalence Equivalence: s t s, t sound a s a t ordering is irrelevant! Eventual consistency reduces to: Same committed operations everywhere All committed operations in every schedule Schedules are sound

Soutenance HDR Gestion répartie d'objets 34 Eventual consistency action a, site i,k, schedule s i : Legal: committed i s i aborted i s i = Safe: s i sound Live: a committed i aborted i a committed i a committed k a aborted i a aborted k

Soutenance HDR Gestion répartie d'objets 35 Global safety invariant sound [ t time, i sites committed i (t) ] Closed for MustHave Non-conflicting: Acyclic in Order Actions successful s, a: a.preCondition (state) Very strong! i commits a at t only if j wont commit conflicting b at t a will succeed everywhere, anytime

Soutenance HDR Gestion répartie d'objets 36 Maintaining global invariant Alternatives: Common knowledge: deterministic abort rule [idem commit?] TWR Unilateral abort [idem commit?] CVS, Holliday 2000 Single primary site decides Bayou, CVS Consensus before deciding Deno, Holliday

Soutenance HDR Gestion répartie d'objets 37 Stability with TWR Independent objects Independent writes (no MustHave nor Order) All sites take same decision: Given two writes to same object, abort the earlier Whether concurrent or not Write stable when seen by all sites Disjointness: committed i = Soundness: no MustHave (no transactions)

Soutenance HDR Gestion répartie d'objets 38 Stability in Bayou Databases: Disjoint Independent: no multi-DB transaction 1 primary / database Log constraints: transactions, time order Disjointness: Only 1 site decides about a: the primary for the database that a updates Soundness: whole transaction commits or aborts

Soutenance HDR Gestion répartie d'objets 39 Hollidays pre-commit protocol Log constraints: multi-object transactions happens-before order Read transactions commit locally Read-Write transactions: consensus to commit convert locks to intentions pre-commit, vote commit if quorum yes abort if anti-quorum no or conflict with committed

Soutenance HDR Gestion répartie d'objets 40 Trade-offs No perfect solution Common knowledge: syntactic: fast, inflexible aborts, doesnt commit Partition + primary single point of failure no MustHave across partition boundaries Consensus slow scalability impossibility of consensus in asynchronous systems with failure

4. Conclusion

Soutenance HDR Gestion répartie d'objets 42 Passage à léchelle ? Réplication en écriture : CC pessimiste = attendre CC optimiste = spéculer Progrès malgré pannes Non transparent Limité par le commit Compromis possibles : partitionner diminuer la granularité limiter nombre décrivains

Soutenance HDR Gestion répartie d'objets 43 Perspectives Importance grandissante du partage Lecture et écriture Commerce électronique Pertinence des techniques Mandataires spécifiques encapsulant létat du protocole Java,.Net : ramasse-miettes réparti Réplication: centres Web, BD Travail déconnecté

Soutenance HDR Gestion répartie d'objets 44 The end