Wiki source code of Clustering

Last modified by Ludovic Dubost on 2023/11/08

Show last authors
1 {{box cssClass="floatinginfobox" title="**Contents**"}}
2 {{toc/}}
3 {{/box}}
4
5 XWiki provides an easy way to setup clustered instances of XWiki based on network events distribution.
6
7 = Features =
8
9 * events synchronization between XWiki instances
10 * multiple clustering channels
11 * possibility to start/stop the clustering channel at runtime
12
13 = Setup =
14
15 == Use a remote Solr instance ==
16
17 XWiki stores a lot of information in Solr cores (e.g. Events, Indexed content), so like with the database you will need to make sure that all cluster members use the [[same Solr instance remotely>>extensions:Extension.Solr Search API#HSetuparemoteSolrserver]] instead of each having its own embedded instance.
18
19 == Enable event distribution ==
20
21 To enable event distribution in an XWiki instance go to ##xwiki.properties## file and set the property ##observation.remote.enabled## to ##true##.
22
23 == Setup communication channels ==
24
25 You will need a configuration file for each different cluster group you want to setup. Generally there is only one for a simple cluster.
26
27 For this you can either:
28
29 * use one of the files provided by jgroups ("tcp", "udp", etc. it's generally enough for most use cases and certainly a lot simpler) which can be customized through system properties
30 * create your own configuration file
31
32 === Create your own configuration file ===
33
34 For this go to the ##/WEB-INF/observation/remote/jgroups## folder and add one xml file.
35
36 See [[JGroups documentation>>http://www.jgroups.org/ug.html]] for more details on how to setup JGroups configuration files.
37
38 If you have IPv6 on your server, you are also advised to read [[this IPv6 article>>https://community.jboss.org/wiki/IPv6]]. Defining ##-Djava.net.preferIPv4Stack=true## when launching the JVM is probably your best bet in most cases. If your really want to use IPv6 for your channels, you should probably upgrade JGroup to version 2.10.0.GA and use a JVM 6 at least.
39
40 == Start communication channels ==
41
42 The name of the xml file matches the identifier of the channel.
43
44 To indicate which channels to start when XWiki starts list them in the property ##observation.remote.channels## of the ##xwiki.properties## file.
45
46 == Choose network adaptor implementation to use ==
47
48 By default only JGroups implementation is provided, but it's possible to add more. See the [[Remote Observation Module>>extensions:Extension.Observation Module Remote#HAddcustomnetworkadaptor]] for more details.
49
50 == Shared filesystem ==
51
52 Attachments and deleted documents located on the file system (located inside the [[Permanent Directory>>Documentation.AdminGuide.Configuration.WebHome#HPermanentDirectory]]) are not clustered and in order for all XWiki instance to access and display them properly in the UI, you'll need to set up a shared filesystem (e.g. NFS) and share the ##store/file## directory.
53
54 Note that you must not share the whole Permanent Directory or this will lead to problems (such as causing file contentions when several instances try to read/write to some of them which have locks on).
55
56 = Limitations =
57
58 The following limitations currently exist when you use clustering:
59
60 * [[XWIKI-6235>>https://jira.xwiki.org/browse/XWIKI-6235]]: The Scheduler feature is not cluster-aware and thus each node of XWiki runs its own scheduler and thus the same scheduler jobs will execute on all nodes. The workaround is to disable the Scheduler plugin on all nodes except on one (i.e remove ##com.xpn.xwiki.plugin.scheduler.SchedulerPlugin## from the ##xwiki.plugins## configuration value in ##xwiki.cfg##). Note that this creates a [[SPOF>>https://en.wikipedia.org/wiki/Single_point_of_failure]] and if the node containing the active Scheduler is down, the scheduled jobs won't execute.
61 * [[XWIKI-14722>>https://jira.xwiki.org/browse/XWIKI-14722]] & [[XWIKI-1516>>https://jira.xwiki.org/browse/XWIKI-15164]]: Attachments and deleted documents located on the file system (located inside the [[Permanent Directory>>Documentation.AdminGuide.Configuration.WebHome#HPermanentDirectory]]) are currently not clustered. See above on setting up a shared filesystem to work around this.
62 * [[XWIKI-11441>>https://jira.xwiki.org/browse/XWIKI-11441]] If a node is down and Events happen, these Events won't be propagated to the node when it comes back up. For example, imagine you install an Extension on a node of the cluster while another node is down. When that node is restarted, the Extension won't be installed on it.
63 * [[XWIKI-15113>>https://jira.xwiki.org/browse/XWIKI-15113]]: Some potential performance issues when saving large documents, as the Annotation updates (updates events and updates the annotations that are impacted by a document content change, to update the selection and context to match the new document content) will run on all nodes in the cluster instead of just once.
64 * [[WEBSOCKET-7>>https://jira.xwiki.org/browse/WEBSOCKET-7]]: Realtime editing currently doesn't work on a cluster. Each node will create a different realtime session, and the collaboration will be with the users that end up on the same cluster node.
65
66 = Performances =
67
68 * You may want to [[configure SOLR to use an external SOLR instance>>xwiki:Documentation.AdminGuide.Performances.WebHome||anchor="HStandaloneSolr"]] instead of having each node maintain its own SOLR index (even though that would work fine but you'd be less performant since each node would do the same work)
69 * See [[general performance page>>Documentation.AdminGuide.Performances]] for more ideas.
70
71 = More =
72
73 See the [[Remote Observation Module>>extensions:Extension.Observation Module Remote]] for more details of the event distribution features and extension capabilities.
74
75 Follow the [[test clustering tutorial>>Documentation.AdminGuide.Clustering.DistributedEventClusterSetup.WebHome]] for a complete tutorial on how to setup a simple cluster between two instances of XWiki on the same server.

Get Connected