ZStack源码剖析之核心库鉴赏——EventFacade与CloudBus

摘要：在注册了相应后，该被调用后则会向相应的发送。开始我们的源码阅读之旅。得到资源所在的在中，很有可能是集群部署的，每个管控不同的资源。则是将统一经过这个队列，同时根据返回给原发送者。则会根据需求改变结果。

本文首发于泊浮目的简书专栏：https://www.jianshu.com/nb/21...

无论是事件和消息驱动，都是解耦的有力手段之一。ZStack作为一个大型软件项目，也使用了这些方案对整个架构进行了解耦。

EventFacade是一个很有意思的组件，因为它几乎是自举的。这就意味着有兴趣的朋友可以copy and paste，然后稍作修改就可以在自己的项目里工作起来了。

在ZStack的repo中，同样提供了相应的case：

package org.zstack.test.core.cloudbus;
/**
 * Created with IntelliJ IDEA.
 * User: frank
 * Time: 12:38 AM
 * To change this template use File | Settings | File Templates.
 */
public class TestCanonicalEvent {
    CLogger logger = Utils.getLogger(TestCanonicalEvent.class);
    ComponentLoader loader;
    EventFacade evtf;
    boolean success;

    @Before
    public void setUp() throws Exception {
        BeanConstructor con = new BeanConstructor();
        loader = con.build();
        evtf = loader.getComponent(EventFacade.class);
        ((EventFacadeImpl) evtf).start();
    }

    @Test
    public void test() throws InterruptedException {
        String path = "/test/event";
        evtf.on(path, new EventRunnable() {
            @Override
            public void run() {
                success = true;
            }
        });

        evtf.fire(path, null);
        TimeUnit.SECONDS.sleep(1);
        Assert.assertTrue(success);
    }
}

使用方法非常简单，先注册一个路径用于接收事件，然后沿着该路径发送一个事件，该事件注册的函数则会被调用。

package org.zstack.core.cloudbus;

import java.util.Map;

/**
 * Created with IntelliJ IDEA.
 * User: frank
 * Time: 11:29 PM
 * To change this template use File | Settings | File Templates.
 */
public interface EventFacade {
    void on(String path, AutoOffEventCallback cb);

    void on(String path, EventCallback cb);

    void on(String path, EventRunnable runnable);

    void off(AbstractEventFacadeCallback cb);

    void onLocal(String path, AutoOffEventCallback cb);

    void onLocal(String path, EventCallback cb);

    void onLocal(String path, EventRunnable runnable);

    void fire(String path, Object data);

    boolean isFromThisManagementNode(Map tokens);

    String META_DATA_MANAGEMENT_NODE_ID = "metadata::managementNodeId";
    String META_DATA_PATH = "metadata::path";
    String WEBHOOK_TYPE = "CanonicalEvent";
}

   @Override
    public void on(String path, AutoOffEventCallback cb) {
        global.put(cb.uniqueIdentity, new CallbackWrapper(path, cb));
    }

    @Override
    public void on(String path, final EventCallback cb) {
        global.put(cb.uniqueIdentity, new CallbackWrapper(path, cb));
    }

    @Override
    public void on(String path, EventRunnable cb) {
        global.put(cb.uniqueIdentity, new CallbackWrapper(path, cb));
    }

on方法仅仅是将一个属于EventRunnable 的uuid作为key，并将Callback作为value放入global这个map中。为什么要这么做呢？因为在Map的key是不可重复的，存path肯定是不妥的。

EventFacadeImpl的方法签名以及成员变量：

public class EventFacadeImpl implements EventFacade, CloudBusEventListener, Component, GlobalApiMessageInterceptor {
    @Autowired
    private CloudBus bus;

    private final Map global = Collections.synchronizedMap(new HashMap<>());
    private final Map local =  new ConcurrentHashMap<>();

    private EventSubscriberReceipt unsubscriber;

相对的fire方法：

    @Override
    public void fire(String path, Object data) {
        assert path != null;
        CanonicalEvent evt = new CanonicalEvent();
        evt.setPath(path);
        evt.setManagementNodeId(Platform.getManagementServerId());
        if (data != null) {
            /*
            if (!TypeUtils.isPrimitiveOrWrapper(data.getClass()) && !data.getClass().isAnnotationPresent(NeedJsonSchema.class)) {
                throw new CloudRuntimeException(String.format("data[%s] passed to canonical event is not annotated by @NeedJsonSchema", data.getClass().getName()));
            }
            */

            evt.setContent(data);
        }
        //从local这个map中找到对应的event并调用
        fireLocal(evt);
        //将事件发送给对应的webhook
        callWebhooks(evt);
        //通过cloudBus发送事件，关于cloudBus的源码之后会讲到
        bus.publish(evt);
    }

在上面的分析中并没有看到global的event是如何被触发的，如果想完全了解其中的过程，还得从CloudBus说起，我们稍后就会提到它。但是已经可以猜到为何要区分on和onLocal了。一个是通过消息总线触发，一个是在当前JVM进程内触发——这意味着一个支持ManagerNode集群事件，一个只支持单个MN事件。这也是来自于ZStack的业务场景——有些事情需要MN一起做，有些事情一个MN做了其他MN就不用做了。介于篇幅，有兴趣的读者可以自行翻看代码，这里不再详举。

WebHook是ZStack向前端主动通信的手段之一。在注册了相应EventPath后，该path被调用后则会向相应的URL发送content。从case——CanonicalEventWebhookCase和WebhookCase可以看到它的正确使用姿势。

class CanonicalEventWebhookCase extends SubCase {
    EnvSpec envSpec

    @Override
    void clean() {
        envSpec.delete()
    }

    @Override
    void setup() {
        INCLUDE_CORE_SERVICES = false
        spring {
            include("webhook.xml")
        }
    }

    String WEBHOOK_PATH = "/canonical-event-webhook"

    void testErrorToCreateWebhookifOpaqueFieldMissing() {
        expect(AssertionError.class) {
            createWebhook {
                name = "webhook1"
                url = "http://127.0.0.1:8989$WEBHOOK_PATH"
                type = EventFacade.WEBHOOK_TYPE
            }
        }
    }

    void testCanonicalEventWithVariableInPath() {
        String path = "/test/{uuid}/event"

        int count = 0
        WebhookInventory hook1 = createWebhook {
            name = "webhook1"
            url = "http://127.0.0.1:8989$WEBHOOK_PATH"
            type = EventFacade.WEBHOOK_TYPE
            opaque = path
        }

        // this webhook will not be called because path unmatching
        WebhookInventory hook2 = createWebhook {
            name = "webhook1"
            url = "http://127.0.0.1:8989$WEBHOOK_PATH"
            type = EventFacade.WEBHOOK_TYPE
            opaque = "/this-path-does-not-match"
        }

        CanonicalEvent evt
        envSpec.simulator(WEBHOOK_PATH) { HttpEntity e ->
            evt = json(e.getBody(), CanonicalEvent.class)
            count ++
            return [:]
        }

        String content = "hello world"
        String eventPath = "/test/${Platform.uuid}/event"
        bean(EventFacade.class).fire(eventPath, content)

        retryInSecs {
            assert count == 1
            assert evt != null
            assert evt.path == eventPath
            assert evt.content == content
            assert evt.managementNodeId == Platform.getManagementServerId()
        }
    }

    void testCanonicalEventUseWebhook() {
        String path = "/test/event"

        WebhookInventory hook1 = createWebhook {
            name = "webhook1"
            url = "http://127.0.0.1:8989$WEBHOOK_PATH"
            type = EventFacade.WEBHOOK_TYPE
            opaque = path
        }

        WebhookInventory hook2 = createWebhook {
            name = "webhook2"
            url = "http://127.0.0.1:8989$WEBHOOK_PATH"
            type = EventFacade.WEBHOOK_TYPE
            opaque = path
        }

        def testFireTwoEvents = {
            List evts = []
            envSpec.simulator(WEBHOOK_PATH) { HttpEntity e ->
                CanonicalEvent evt = json(e.getBody(), CanonicalEvent.class)
                evts.add(evt)
                return [:]
            }

            String content = "hello world"
            bean(EventFacade.class).fire(path, content)

            retryInSecs {
                assert evts.size() == 2
                CanonicalEvent evt1 = evts[0]
                CanonicalEvent evt2 = evts[1]
                assert evt1.path == path
                assert evt1.content == content
                assert evt1.managementNodeId == Platform.getManagementServerId()
                assert evt2.path == path
                assert evt2.content == content
                assert evt2.managementNodeId == Platform.getManagementServerId()
            }
        }

        def testOneEventsGetAfterDeleteOneHook = {
            deleteWebhook { uuid = hook1.uuid }

            List evts = []
            envSpec.simulator(WEBHOOK_PATH) { HttpEntity e ->
                CanonicalEvent evt = json(e.getBody(), CanonicalEvent.class)
                evts.add(evt)
                return [:]
            }

            String content = "hello world"
            bean(EventFacade.class).fire(path, content)

            retryInSecs {
                assert evts.size() == 1
            }
        }

        def testNoEventGetAfterDeleteAllHooks = {
            deleteWebhook { uuid = hook2.uuid }

            List evts = []
            envSpec.simulator(WEBHOOK_PATH) { HttpEntity e ->
                CanonicalEvent evt = json(e.getBody(), CanonicalEvent.class)
                evts.add(evt)
                return [:]
            }

            String content = "hello world"
            bean(EventFacade.class).fire(path, content)

            retryInSecs {
                assert evts.size() == 0
            }
        }

        testFireTwoEvents()
        testOneEventsGetAfterDeleteOneHook()
        testNoEventGetAfterDeleteAllHooks()
    }

    @Override
    void environment() {
        envSpec = env {
            // nothing
        }
    }

    @Override
    void test() {
        envSpec.create {
            testCanonicalEventUseWebhook()
            testCanonicalEventWithVariableInPath()
            testErrorToCreateWebhookifOpaqueFieldMissing()
        }
    }
}

class WebhookCase extends SubCase {
    EnvSpec envSpec

    @Override
    void clean() {
        envSpec.delete()
    }

    @Override
    void setup() {
        INCLUDE_CORE_SERVICES = false
        spring {
            include("webhook.xml")
        }
    }

    @Override
    void environment() {
        envSpec = env {
            // nothing
        }
    }

    void testWebhooksCRUD() {
        WebhookInventory hook = null

        def testCreateWebhook = {
            def params = null

            hook = createWebhook {
                name = "webhook"
                type = "custom-type"
                url = "http://127.0.0.1:8080/webhook"
                description = "desc"
                opaque = "test data"

                params = delegate
            }

            assert dbIsExists(hook.uuid, WebhookVO.class)
            assert hook.name == params.name
            assert hook.type == params.type
            assert hook.url == params.url
            assert hook.description == params.description
            assert hook.opaque == params.opaque
        }

        def testQueryWebhook = {
            List invs = queryWebhook {
                conditions = ["name=${hook.name}"]
            }

            assert invs.size() == 1
            assert invs[0].uuid == hook.uuid
        }

        def testDeleteWebhook = {
            deleteWebhook {
                uuid = hook.uuid
            }

            assert !dbIsExists(hook.uuid, WebhookVO.class)
        }

        testCreateWebhook()
        testQueryWebhook()
        testDeleteWebhook()
    }

    void testInvalidUrl() {
        expect(AssertionError.class) {
            createWebhook {
                name = "webhook"
                type = "custom-type"
                url = "this is not a url"
                description = "desc"
                opaque = "test data"
            }
        }
    }

    @Override
    void test() {
        envSpec.create {
            testWebhooksCRUD()
            testInvalidUrl()
        }
    }
}

CloudBus可以说是ZStack中最重要的组件了，ZStack各个模块的通信全部是由Message来完成的，而CloudBus就是它们的通信媒介，接下来我们来看它的源码。

本节适合对AMQP有一定了解同学，如果不了解可以先看我的博客MQ学习小记

先看一个相关的Case：

package org.zstack.test.core.cloudbus;

import junit.framework.Assert;
import org.junit.Before;
import org.junit.Test;
import org.zstack.core.cloudbus.CloudBusIN;
import org.zstack.core.componentloader.ComponentLoader;
import org.zstack.header.AbstractService;
import org.zstack.header.Service;
import org.zstack.header.message.Message;
import org.zstack.header.message.MessageReply;
import org.zstack.header.message.NeedReplyMessage;
import org.zstack.test.BeanConstructor;
import org.zstack.utils.Utils;
import org.zstack.utils.logging.CLogger;

import java.util.concurrent.TimeUnit;

public class TestCloudBusCall {
    CLogger logger = Utils.getLogger(TestCloudBusCall.class);
    ComponentLoader loader;
    CloudBusIN bus;
    Service serv;

    public static class HelloWorldMsg extends NeedReplyMessage {
        private String greet;

        public String getGreet() {
            return greet;
        }

        public void setGreet(String greet) {
            this.greet = greet;
        }

    }

    public static class HelloWorldReply extends MessageReply {
        private String greet;

        public String getGreet() {
            return greet;
        }

        public void setGreet(String greet) {
            this.greet = greet;
        }
    }

    class FakeService extends AbstractService {
        @Override
        public boolean start() {
            bus.registerService(this);
            bus.activeService(this);
            return true;
        }

        @Override
        public boolean stop() {
            bus.deActiveService(this);
            bus.unregisterService(this);
            return true;
        }

        @Override
        public void handleMessage(Message msg) {
            if (msg.getClass() == HelloWorldMsg.class) {
                HelloWorldMsg hmsg = (HelloWorldMsg) msg;
                HelloWorldReply r = new HelloWorldReply();
                r.setGreet(hmsg.getGreet());
                bus.reply(msg, r);
            }
        }

        @Override
        public String getId() {
            return this.getClass().getCanonicalName();
        }

    }

    @Before
    public void setUp() throws Exception {
        BeanConstructor con = new BeanConstructor();
        loader = con.build();
        bus = loader.getComponent(CloudBusIN.class);
        serv = new FakeService();
        serv.start();
    }

    @Test
    public void test() throws InterruptedException, ClassNotFoundException {
        HelloWorldMsg msg = new HelloWorldMsg();
        msg.setGreet("Hello");
        msg.setServiceId(FakeService.class.getCanonicalName());
        msg.setTimeout(TimeUnit.SECONDS.toMillis(10));
        HelloWorldReply r = (HelloWorldReply) bus.call(msg);
        serv.stop();
        Assert.assertEquals("Hello", r.getGreet());
    }
}
我们注册了一个Service，并覆写HandleMessage方法，在Case中，我们成功收到了消息并通过了断言。


再看一个：

package org.zstack.test.core.cloudbus;

import junit.framework.Assert;
import org.junit.Before;
import org.junit.Test;
import org.zstack.core.cloudbus.CloudBusCallBack;
import org.zstack.core.cloudbus.CloudBusIN;
import org.zstack.core.componentloader.ComponentLoader;
import org.zstack.header.AbstractService;
import org.zstack.header.Service;
import org.zstack.header.message.Message;
import org.zstack.header.message.MessageReply;
import org.zstack.header.message.NeedReplyMessage;
import org.zstack.test.BeanConstructor;
import org.zstack.utils.Utils;
import org.zstack.utils.logging.CLogger;

import java.util.concurrent.CountDownLatch;
import java.util.concurrent.TimeUnit;

public class TestCloudBusSendCallback {

CLogger logger = Utils.getLogger(TestCloudBusSendCallback.class);
ComponentLoader loader;
CloudBusIN bus;
CountDownLatch latch = new CountDownLatch(1);
boolean isSuccess = false;
Service serv;

public static class HelloWorldMsg extends NeedReplyMessage {
    private String greet;

    public String getGreet() {
        return greet;
    }

    public void setGreet(String greet) {
        this.greet = greet;
    }

}

public static class HelloWorldReply extends MessageReply {
    private String greet;

    public String getGreet() {
        return greet;
    }

    public void setGreet(String greet) {
        this.greet = greet;
    }
}

class FakeService extends AbstractService {
    @Override
    public boolean start() {
        bus.registerService(this);
        bus.activeService(this);
        return true;
    }

    @Override
    public boolean stop() {
        bus.deActiveService(this);
        bus.unregisterService(this);
        return true;
    }

    @Override
    public void handleMessage(Message msg) {
        if (msg.getClass() == HelloWorldMsg.class) {
            HelloWorldMsg hmsg = (HelloWorldMsg) msg;
            HelloWorldReply r = new HelloWorldReply();
            r.setGreet(hmsg.getGreet());
            bus.reply(msg, r);
        }
    }

    @Override
    public String getId() {
        return this.getClass().getCanonicalName();
    }

}

@Before
public void setUp() throws Exception {
    BeanConstructor con = new BeanConstructor();
    loader = con.build();
    bus = loader.getComponent(CloudBusIN.class);
    serv = new FakeService();
    serv.start();
}

@Test
public void test() throws InterruptedException, ClassNotFoundException {
    HelloWorldMsg msg = new HelloWorldMsg();
    msg.setGreet("Hello");
    msg.setServiceId(FakeService.class.getCanonicalName());
    msg.setTimeout(TimeUnit.SECONDS.toMillis(10));
    bus.send(msg, new CloudBusCallBack(null) {
        @Override
        public void run(MessageReply reply) {
            if (reply instanceof HelloWorldReply) {
                HelloWorldReply hr = (HelloWorldReply) reply;
                if ("Hello".equals(hr.getGreet())) {
                    isSuccess = true;
                }
            }
            latch.countDown();
        }
    });
    latch.await(15, TimeUnit.SECONDS);
    serv.stop();
    Assert.assertEquals(true, isSuccess);
}

}

同样也是注册了一个Service，然后使用了CallBack，如果运行一下发现断言是可以通过的——意味着CallBack有被调用。

综上，使用CloudBus很简单——只需要注册你的Service，使用CloudBus指定Service发送，Service就能收到，如果你需要注册你的CallBack，也能很简单完成。

这么好用的东西，内部实现恐怕不会太简单。我们先从接口开始看：

package org.zstack.core.cloudbus;

import org.zstack.header.Component;
import org.zstack.header.Service;
import org.zstack.header.errorcode.ErrorCode;
import org.zstack.header.exception.CloudConfigureFailException;
import org.zstack.header.message.*;

import java.util.List;

public interface CloudBus extends Component {
    void send(Message msg);
    
     void send(List msgs);
    
    void send(NeedReplyMessage msg, CloudBusCallBack callback);

    @Deprecated
    void send(List msgs, CloudBusListCallBack callBack);

    @Deprecated
    void send(List msgs, int parallelLevel, CloudBusListCallBack callBack);

    @Deprecated
    void send(List msgs, int parallelLevel, CloudBusSteppingCallback callback);

    void route(List msgs);
    
    void route(Message msg);
    
    void reply(Message request, MessageReply reply);
    
    void publish(List events);
    
    void publish(Event event);
    
    MessageReply call(NeedReplyMessage msg);
    
     List call(List msg);
    
    void registerService(Service serv) throws CloudConfigureFailException;
    
    void unregisterService(Service serv);
    
    EventSubscriberReceipt subscribeEvent(CloudBusEventListener listener, Event...events);
    
    void dealWithUnknownMessage(Message msg);
    
    void replyErrorByMessageType(Message msg, Exception e);
    
    void replyErrorByMessageType(Message msg, String err);
    
    void replyErrorByMessageType(Message msg, ErrorCode err);
    
    void logExceptionWithMessageDump(Message msg, Throwable e);
    
    String makeLocalServiceId(String serviceId);

    void makeLocalServiceId(Message msg, String serviceId);

    String makeServiceIdByManagementNodeId(String serviceId, String managementNodeId);

    void makeServiceIdByManagementNodeId(Message msg, String serviceId, String managementNodeId);

    String makeTargetServiceIdByResourceUuid(String serviceId, String resourceUuid);

    void makeTargetServiceIdByResourceUuid(Message msg, String serviceId, String resourceUuid);

    void installBeforeDeliveryMessageInterceptor(BeforeDeliveryMessageInterceptor interceptor, Class...classes);

    void installBeforeSendMessageInterceptor(BeforeSendMessageInterceptor interceptor, Class...classes);

    void installBeforePublishEventInterceptor(BeforePublishEventInterceptor interceptor, Class...classes);
}

接口的命名语义较为清晰，在这里不多做解释。开始我们的源码阅读之旅。

init是在bean处于加载器，Spring提供的一个钩子。在xml中我们可以看到声明：

init方法：

    void init() {
        trackerClose = CloudBusGlobalProperty.CLOSE_TRACKER;
        serverIps = CloudBusGlobalProperty.SERVER_IPS;
        tracker = new MessageTracker();

        ConnectionFactory connFactory = new ConnectionFactory();
        List
 addresses = CollectionUtils.transformToList(serverIps, new Function() {
            @Override
            public Address call(String arg) {
                return Address.parseAddress(arg);
            }
        });
        connFactory.setAutomaticRecoveryEnabled(true);
        connFactory.setRequestedHeartbeat(CloudBusGlobalProperty.RABBITMQ_HEART_BEAT_TIMEOUT);
        connFactory.setNetworkRecoveryInterval((int) TimeUnit.SECONDS.toMillis(CloudBusGlobalProperty.RABBITMQ_NETWORK_RECOVER_INTERVAL));
        connFactory.setConnectionTimeout((int) TimeUnit.SECONDS.toMillis(CloudBusGlobalProperty.RABBITMQ_CONNECTION_TIMEOUT));

        logger.info(String.format("use RabbitMQ server IPs: %s", serverIps));

        try {
            if (CloudBusGlobalProperty.RABBITMQ_USERNAME != null) {
                connFactory.setUsername(CloudBusGlobalProperty.RABBITMQ_USERNAME);
                logger.info(String.format("use RabbitMQ username: %s", CloudBusGlobalProperty.RABBITMQ_USERNAME));
            }
            if (CloudBusGlobalProperty.RABBITMQ_PASSWORD != null) {
                connFactory.setPassword(CloudBusGlobalProperty.RABBITMQ_PASSWORD);
                logger.info("use RabbitMQ password: ******");
            }
            if (CloudBusGlobalProperty.RABBITMQ_VIRTUAL_HOST != null) {
                connFactory.setVirtualHost(CloudBusGlobalProperty.RABBITMQ_VIRTUAL_HOST);
                logger.info(String.format("use RabbitMQ virtual host: %s", CloudBusGlobalProperty.RABBITMQ_VIRTUAL_HOST));
            }

            conn = connFactory.newConnection(addresses.toArray(new Address[]{}));
            logger.debug(String.format("rabbitmq connection is established on %s", conn.getAddress()));

            ((Recoverable)conn).addRecoveryListener(new RecoveryListener() {
                @Override
                public void handleRecovery(Recoverable recoverable) {
                    logger.info(String.format("rabbitmq connection is recovering on %s", conn.getAddress().toString()));
                }
            });

            channelPool = new ChannelPool(CloudBusGlobalProperty.CHANNEL_POOL_SIZE, conn);
            createExchanges();
            outboundQueue = new BusQueue(makeMessageQueueName(SERVICE_ID), BusExchange.P2P);
            Channel chan = channelPool.acquire();
            chan.queueDeclare(outboundQueue.getName(), false, false, true, queueArguments());
            chan.basicConsume(outboundQueue.getName(), true, consumer);
            chan.queueBind(outboundQueue.getName(), outboundQueue.getBusExchange().toString(), outboundQueue.getBindingKey());
            channelPool.returnChannel(chan);
            maid.construct();
            noRouteEndPoint.construct();
            tracker.construct();
            tracker.trackService(SERVICE_ID);
        } catch (Exception e) {
            throw new CloudRuntimeException(e);
        }
    }

简单来说，该函数尝试获取配置文件中与RabbitMQ中相关的配置，并初始化Connection，并以此为基础创建了channel poll。然后将一个channel和一个messageQueue绑定在了一起。同时构造了EventMaid和noRouteEndPoint和tracker，后二者都是Message的消费者，看名字就可以看出来，一个用于订阅/发布模型（绑定此交换器的队列都会收到消息），一个用于track。

start则是ZStack定义的一个钩子，当ManagerNode起来的时候，start会被调用到。

   @Override
    public boolean start() {
        populateExtension();
        prepareStatistics();

        for (Service serv : services) {
            assert serv.getId() != null : String.format("service id can not be null[%s]", serv.getClass().getName());
            registerService(serv);
        }

        jmxf.registerBean("CloudBus", this);

        return true;
    }

一个个看：

    private void populateExtension() {
        services = pluginRgty.getExtensionList(Service.class);
        for (ReplyMessagePreSendingExtensionPoint extp : pluginRgty.getExtensionList(ReplyMessagePreSendingExtensionPoint.class)) {
            List clazzs = extp.getReplyMessageClassForPreSendingExtensionPoint();
            if (clazzs == null || clazzs.isEmpty()) {
                continue;
            }

            for (Class clz : clazzs) {
                if (!(APIEvent.class.isAssignableFrom(clz)) && !(MessageReply.class.isAssignableFrom(clz))) {
                    throw new CloudRuntimeException(String.format("ReplyMessagePreSendingExtensionPoint can only marshal APIEvent or MessageReply. %s claimed by %s is neither APIEvent nor MessageReply",
                            clz.getName(), extp.getClass().getName()));
                }

                List exts = replyMessageMarshaller.get(clz);
                if (exts == null) {
                    exts = new ArrayList();
                    replyMessageMarshaller.put(clz, exts);
                }
                exts.add(extp);
            }
        }
    }

首先收集了所有继承于Service的类，然后加载会改变msg reply的extensionPoint。

 private void prepareStatistics() {
        List needReplyMsgs = BeanUtils.scanClassByType("org.zstack", NeedReplyMessage.class);
        needReplyMsgs = CollectionUtils.transformToList(needReplyMsgs, new Function() {
            @Override
            public Class call(Class arg) {
                return !APIMessage.class.isAssignableFrom(arg) || APISyncCallMessage.class.isAssignableFrom(arg) ? arg : null;
            }
        });

        for (Class clz : needReplyMsgs) {
            MessageStatistic stat = new MessageStatistic();
            stat.setMessageClassName(clz.getName());
            statistics.put(stat.getMessageClassName(), stat);
        }
    }

为需要回复的msg设置统计信息。

之后就是把所有的Service收集起来，方便Msg的分发。

    @Override
    public String makeLocalServiceId(String serviceId) {
        return serviceId + "." + Platform.getManagementServerId();
    }

    @Override
    public void makeLocalServiceId(Message msg, String serviceId) {
        msg.setServiceId(makeLocalServiceId(serviceId));
    }

如ZStack的伸缩性秘密武器：无状态服务中所说一般，每个管理节点都会注册一堆服务队列。因此我们要按照其格式组装，这样消息才能被服务所接收。

    @Override
    public String makeTargetServiceIdByResourceUuid(String serviceId, String resourceUuid) {
        DebugUtils.Assert(serviceId!=null, "serviceId cannot be null");
        DebugUtils.Assert(resourceUuid!=null, "resourceUuid cannot be null");
        //得到资源所在的MN UUID
        String mgmtUuid = destMaker.makeDestination(resourceUuid);
        return serviceId + "." + mgmtUuid;
    }

    @Override
    public void makeTargetServiceIdByResourceUuid(Message msg, String serviceId, String resourceUuid) {
        String targetService = makeTargetServiceIdByResourceUuid(serviceId, resourceUuid);
        msg.setServiceId(targetService);
    }

在ZStack中，ManagerNode很有可能是集群部署的，每个MN管控不同的资源。那么就需要一致性哈希环来确定资源所在哪个MN。

    @Override
    public void send(final NeedReplyMessage msg, final CloudBusCallBack callback) {
        //给msg一个超时时间
        evaluateMessageTimeout(msg);
        //new继承于Envelope的匿名内部类
        Envelope e = new Envelope() {
            //用来判断这个msg是否已经发出去了
            AtomicBoolean called = new AtomicBoolean(false);

            final Envelope self = this;
            //计算超时，往线程池提交一个任务
            TimeoutTaskReceipt timeoutTaskReceipt = thdf.submitTimeoutTask(new Runnable() {
                @Override
                public void run() {
                    self.timeout();
                }
            }, TimeUnit.MILLISECONDS, msg.getTimeout());

            @Override
            //msg 发送成功时候调用这个方法
            public void ack(MessageReply reply) {
                //计算该msg耗时
                count(msg);
                //根据msg的唯一UUID移除在这个map中的记录
                envelopes.remove(msg.getId());
                //如果更新失败，说明这个消息已经被发送过了。返回
                if (!called.compareAndSet(false, true)) {
                    return;
                }
                //取消一个计算超时的任务
                timeoutTaskReceipt.cancel();
                //调用注册的callback
                callback.run(reply);
            }

            //消息超时时调用的逻辑
            @Override
            public void timeout() {
                // 根据msg的唯一UUID移除在这个map中的记录
                envelopes.remove(msg.getId());
                 // 如何已经被调用过则返回
                if (!called.compareAndSet(false, true)) {
                    return;
                }
                //内部构造一个超时reply返回给callback
                callback.run(createTimeoutReply(msg));
            }
            //用于getWaitingReplyMessageStatistic
            @Override
            List getRequests() {
                List requests = new ArrayList();
                requests.add(msg);
                return requests;
            }
        };
        
        //往envelopes这个map里放入msg的唯一UUID和刚刚构造的envelope
        envelopes.put(msg.getId(), e);
        //发送消息
        send(msg, false);
    }

    private void send(Message msg, Boolean noNeedReply) {
        //msg的serviceID不允许为空，不然不能
        if (msg.getServiceId() == null) {
            throw new IllegalArgumentException(String.format("service id cannot be null: %s", msg.getClass().getName()));
        }
        //为msg构建基本属性
        basicProperty(msg);
        //设置msg header属性
        msg.putHeaderEntry(CORRELATION_ID, msg.getId());
        //消息的回复队列设置
        msg.putHeaderEntry(REPLY_TO, outboundQueue.getBindingKey());
        if (msg instanceof APIMessage) {
            // API always need reply
            msg.putHeaderEntry(NO_NEED_REPLY_MSG, Boolean.FALSE.toString());
        } else if (msg instanceof NeedReplyMessage) {
            // for NeedReplyMessage sent without requiring receiver to reply,
            // mark it, then it will not be tracked and replied
            msg.putHeaderEntry(NO_NEED_REPLY_MSG, noNeedReply.toString());
        }

        buildRequestMessageMetaData(msg);
        wire.send(msg);
    }

该函数是一段公用逻辑。所有的消息都是从这里进去然后由rabbitMQ发出去的。所以在这里需要多说几句。

    protected void basicProperty(Message msg) {
        AMQP.BasicProperties.Builder builder = new AMQP.BasicProperties.Builder();
        msg.setAMQPProperties(builder.deliveryMode(1).expiration(String.valueOf(TimeUnit.SECONDS.toMillis(CloudBusGlobalProperty.MESSAGE_TTL))).build());
    }

这个函数设置了msg基础属性——持久化策略（否）和超时。

那么再看buildRequestMessageMetaData方法

    private void buildRequestMessageMetaData(Message msg) {
        if (msg instanceof APIMessage || (msg instanceof NeedReplyMessage && !Boolean.valueOf((String)msg.getHeaderEntry(NO_NEED_REPLY_MSG)))) {
            RequestMessageMetaData metaData;
            if (msg instanceof LockResourceMessage) {
                LockResourceMessage lmsg = (LockResourceMessage) msg;
                LockMessageMetaData lmetaData = new LockMessageMetaData();
                lmetaData.unlockKey = lmsg.getUnlockKey();
                lmetaData.reason = lmsg.getReason();
                lmetaData.senderManagementUuid = Platform.getManagementServerId();
                metaData = lmetaData;
            } else {
                metaData = new RequestMessageMetaData();
            }

            metaData.needApiEvent = msg instanceof APIMessage && !(msg instanceof APISyncCallMessage);
            metaData.msgId = msg.getId();
            metaData.replyTo = msg.getHeaderEntry(REPLY_TO);
            metaData.timeout = msg instanceof NeedReplyMessage ? ((NeedReplyMessage) msg).getTimeout() : null;
            metaData.serviceId = msg.getServiceId();
            metaData.messageName = msg.getClass().getName();
            metaData.className = metaData.getClass().getName();
            msg.getAMQPHeaders().put(MESSAGE_META_DATA, JSONObjectUtil.toJsonString(metaData));
        }
    }

方法buildRequestMessageMetaData将消息所需的metaData从msg里取了出来并放入了msg的真正Header中。

然后是wire.send:

        public void send(Message msg) {
            // for unit test finding invocation chain
            MessageCommandRecorder.record(msg.getClass());

            List interceptors = beforeSendMessageInterceptors.get(msg.getClass());
            if (interceptors != null) {
                for (BeforeSendMessageInterceptor interceptor : interceptors) {
                    interceptor.intercept(msg);

                    /*
                    if (logger.isTraceEnabled()) {
                        logger.trace(String.format("called %s for message[%s]", interceptor.getClass(), msg.getClass()));
                    }
                    */
                }
            }

            for (BeforeSendMessageInterceptor interceptor : beforeSendMessageInterceptorsForAll) {
                interceptor.intercept(msg);

                /*
                if (logger.isTraceEnabled()) {
                    logger.trace(String.format("called %s for message[%s]", interceptor.getClass(), msg.getClass()));
                }
                */
            }

            send(msg, true);
        }

逻辑一目了然：

记录它的调用链

调用它专属的发送前拦截器进行拦截

调用所有msg的发送前拦截器进行拦截

send(msg, true);:

       public void send(final Message msg, boolean makeQueueName) {
            /*
            StopWatch watch = new StopWatch();
            watch.start();
            */
            String serviceId = msg.getServiceId();
            if (makeQueueName) { 
                //获取真正的队列名
                serviceId = makeMessageQueueName(serviceId);
            }
            // build json schema
            buildSchema(msg);
            //当前的thread Context中获取必要信息。每个api调用所携带的uuid就是这样传递下去的
            evalThreadContextToMessage(msg);

            if (logger.isTraceEnabled() && logMessage(msg)) {
                logger.trace(String.format("[msg send]: %s", wire.dumpMessage(msg)));
            }

            //从channel poll 中取出一个channel 
            Channel chan = channelPool.acquire();
            try {
                //接下来多带带解释
                new RecoverableSend(chan, msg, serviceId, outboundQueue.getBusExchange()).send();
                /*
                watch.stop();
                logger.debug(String.mediaType("sending %s cost %sms", msg.getClass().getName(), watch.getTime()));
                */
            } catch (IOException e) {
                throw new CloudRuntimeException(e);
            } finally {
                //返回给channel poll
                channelPool.returnChannel(chan);
            }
        }

多带带分析 new RecoverableSend(chan, msg, serviceId, outboundQueue.getBusExchange()).send();：

        private class RecoverableSend {
            Channel chan;
            byte[] data;
            String serviceId;
            Message msg;
            BusExchange exchange;

            RecoverableSend(Channel chan, Message msg, String serviceId, BusExchange exchange) throws IOException {
                data = compressMessageIfNeeded(msg);
                this.chan = chan;
                this.serviceId = serviceId;
                this.msg = msg;
                this.exchange = exchange;
            }

            void send() throws IOException {
                try {
                    chan.basicPublish(exchange.toString(), serviceId,
                            true, msg.getAMQPProperties(), data);
                } catch (ShutdownSignalException e) {
                    if (!(conn instanceof AutorecoveringConnection) || serverIps.size() <= 1 || !Platform.IS_RUNNING) {
                        // the connection is not recoverable
                        throw e;
                    }

                    logger.warn(String.format("failed to send a message because %s; as the connection is recoverable," +
                            "we are doing recoverable send right now", e.getMessage()));

                    if (!recoverSend()) {
                        throw e;
                    }
                }
            }

            private byte[] compressMessageIfNeeded(Message msg) throws IOException {
                if (!CloudBusGlobalProperty.COMPRESS_NON_API_MESSAGE || msg instanceof APIEvent || msg instanceof APIMessage) {
                    return gson.toJson(msg, Message.class).getBytes();
                }

                msg.getAMQPHeaders().put(AMQP_PROPERTY_HEADER__COMPRESSED, "true");
                return Compresser.deflate(gson.toJson(msg, Message.class).getBytes());
            }

            private boolean recoverSend() throws IOException {
                int interval = conn.getHeartbeat() / 2;
                interval = interval > 0 ? interval : 1;
                int count = 0;

                // as the connection is lost, there is no need to wait heart beat missing 8 times
                // so we use reflection to fast the process
                RecoveryAwareAMQConnection delegate = FieldUtils.getFieldValue("delegate", conn);
                DebugUtils.Assert(delegate != null, "cannot get RecoveryAwareAMQConnection");
                Field _missedHeartbeats = FieldUtils.getField("_missedHeartbeats", RecoveryAwareAMQConnection.class);
                DebugUtils.Assert(_missedHeartbeats!=null, "cannot find _missedHeartbeats");
                _missedHeartbeats.setAccessible(true);
                try {
                    _missedHeartbeats.set(delegate, 100);
                } catch (IllegalAccessException e) {
                    throw new CloudRuntimeException(e);
                }

                while (count < CloudBusGlobalProperty.RABBITMQ_RECOVERABLE_SEND_TIMES) {
                    try {
                        TimeUnit.SECONDS.sleep(interval);
                    } catch (InterruptedException e1) {
                        logger.warn(e1.getMessage());
                    }

                    try {
                        chan.basicPublish(exchange.toString(), serviceId,
                                true, msg.getAMQPProperties(), data);
                        return true;
                    } catch (ShutdownSignalException e) {
                        logger.warn(String.format("recoverable send fails %s times, will continue to retry %s times; %s",
                                count, CloudBusGlobalProperty.RABBITMQ_RECOVERABLE_SEND_TIMES-count, e.getMessage()));
                        count ++;
                    }
                }

                return false;
            }
        }

最核心的代码即是：

                    chan.basicPublish(exchange.toString(), serviceId,
                            true, msg.getAMQPProperties(), data);

根据交换器、绑定器的key和msg的基本属性还有已经序列化的msg在RabbitMQ中发送消息。

我们可以看一下该方法签名：

    /**
     * Publish a message
     * @see com.rabbitmq.client.AMQP.Basic.Publish
     * @param exchange the exchange to publish the message to
     * @param routingKey the routing key
     * @param mandatory true if the "mandatory" flag is to be set
     * @param props other properties for the message - routing headers etc
     * @param body the message body
     * @throws java.io.IOException if an error is encountered
     */
    void basicPublish(String exchange, String routingKey, boolean mandatory, BasicProperties props, byte[] body)
            throws IOException;

当mandatory标志位设置为true时，如果exchange根据自身类型和消息routeKey无法找到一个符合条件的queue，那么会调用basic.return方法将消息返还给生产者；当mandatory设为false时，出现上述情形broker会直接将消息扔掉。

还有一个附有immediate的方法：

    /**
     * Publish a message
     * @see com.rabbitmq.client.AMQP.Basic.Publish
     * @param exchange the exchange to publish the message to
     * @param routingKey the routing key
     * @param mandatory true if the "mandatory" flag is to be set
     * @param immediate true if the "immediate" flag is to be
     * set. Note that the RabbitMQ server does not support this flag.
     * @param props other properties for the message - routing headers etc
     * @param body the message body
     * @throws java.io.IOException if an error is encountered
     */
    void basicPublish(String exchange, String routingKey, boolean mandatory, boolean immediate, BasicProperties props, byte[] body)
            throws IOException;

当immediate标志位设置为true时，如果exchange在将消息route到queue(s)时发现对应的queue上没有消费者，那么这条消息不会放入队列中。当与消息routeKey关联的所有queue(一个或多个)都没有消费者时，该消息会通过basic.return方法返还给生产者。

    @Override
    public void reply(Message request, MessageReply reply) {
        if (Boolean.valueOf((String) request.getHeaderEntry(NO_NEED_REPLY_MSG))) {
            if (logger.isTraceEnabled()) {
                logger.trace(String.format("%s in message%s is set, drop reply%s", NO_NEED_REPLY_MSG,
                        wire.dumpMessage(request), wire.dumpMessage(reply)));
            }

            return;
        }

        AMQP.BasicProperties.Builder builder = new AMQP.BasicProperties.Builder();
        reply.setAMQPProperties(builder.deliveryMode(1).build());
        reply.getHeaders().put(IS_MESSAGE_REPLY, Boolean.TRUE.toString());
        reply.putHeaderEntry(CORRELATION_ID, request.getId());
        reply.setServiceId((String) request.getHeaderEntry(REPLY_TO));

        buildResponseMessageMetaData(reply);
        if (request instanceof NeedReplyMessage) {
            callReplyPreSendingExtensions(reply, (NeedReplyMessage) request);
        }
        wire.send(reply, false);
    }

其他属性之前都有提到。 reply.setServiceId((String) request.getHeaderEntry(REPLY_TO));则是将reply统一经过outboundQueue这个队列，同时根据correlationId返回给原发送者。

callReplyPreSendingExtensions则会根据需求改变reply结果。之后就是wire.send，之前已经分析过了。

    @Override
    public void publish(Event event) {
        if (event instanceof APIEvent) {
            APIEvent aevt = (APIEvent) event;
            DebugUtils.Assert(aevt.getApiId() != null, String.format("apiId of %s cannot be null", aevt.getClass().getName()));
        }
        //和前面的msgProperty一样
        eventProperty(event);
        //构建metaData
        buildResponseMessageMetaData(event);
        //前面分析过了
        callReplyPreSendingExtensions(event, null);
        //调用beforeEventPublishInterceptors。为了抛出异常的时候方便track，声明了这样的一个变量。
        BeforePublishEventInterceptor c = null;
        try {
            List is = beforeEventPublishInterceptors.get(event.getClass());
            if (is != null) {
                for (BeforePublishEventInterceptor i : is) {
                    c = i;
                    i.beforePublishEvent(event);
                }
            }

            for (BeforePublishEventInterceptor i : beforeEventPublishInterceptorsForAll)  {
                c = i;
                i.beforePublishEvent(event);
            }
        } catch (StopRoutingException e) {
            if (logger.isTraceEnabled()) {
                logger.trace(String.format("BeforePublishEventInterceptor[%s] stop publishing event: %s",
                        c == null ? "null" : c.getClass().getName(), JSONObjectUtil.toJsonString(event)));
            }

            return;
        }

        wire.publish(event);
    }

接下来看wire.publish方法

        public void publish(Event evt) {
            /*
            StopWatch watch = new StopWatch();
            watch.start();
            */

            buildSchema(evt);

            evalThreadContextToMessage(evt);

            if (logger.isTraceEnabled() && logMessage(evt)) {
                logger.trace(String.format("[event publish]: %s", wire.dumpMessage(evt)));
            }

            Channel chan = channelPool.acquire();
            try {
                new RecoverableSend(chan, evt, evt.getType().toString(), BusExchange.BROADCAST).send();
                /*
                watch.stop();
                logger.debug(String.mediaType("sending %s cost %sms", evt.getClass().getName(), watch.getTime()));
                */
            } catch (IOException e) {
                throw new CloudRuntimeException(e);
            } finally {
                channelPool.returnChannel(chan);
            }
        }

大部分方法和send无异。但是在Event的类中定义了两种Type：

package org.zstack.header.message;

import org.zstack.header.rest.APINoSee;

public abstract class Event extends Message {
    /**
     * @ignore
     */
    @APINoSee
    private String avoidKey;

    public String getAvoidKey() {
        return avoidKey;
    }

    public void setAvoidKey(String avoidKey) {
        this.avoidKey = avoidKey;
    }

    public abstract Type getType();

    public abstract String getSubCategory();

    public static final String BINDING_KEY_PERFIX = "key.event.";

    public static enum Category {
        LOCAL,
        API,
    }

    public static class Type {
        private final String _name;

        public Type(Category ctg, String subCtg) {
            _name = BINDING_KEY_PERFIX + ctg.toString() + "." + subCtg;
        }

        @Override
        public String toString() {
            return _name;
        }

        @Override
        public int hashCode() {
            return _name.hashCode();
        }

        @Override
        public boolean equals(Object t) {
            if (!(t instanceof Type)) {
                return false;
            }

            Type type = (Type) t;
            return _name.equals(type.toString());
        }
    }
}

即Local和API。从名字上很好看出来，一个用来回复APIMsg的，一个用来发布本地消息。不过要了解这里面的细节，就得看EventMaid了。

EventMaid

    private class EventMaid extends AbstractConsumer {
        Map> listeners = new ConcurrentHashMap>();
        Channel eventChan;
        String queueName = makeEventQueueName(String.format("eventMaid.%s", Platform.getUuid()));

        public void construct() {
            try {
                eventChan = conn.createChannel();
                eventChan.queueDeclare(queueName, false, false, true, queueArguments());
                eventChan.basicConsume(queueName, true, this);
            } catch (IOException e) {
                throw new CloudRuntimeException(e);
            }
        }

        public void destruct() {
            try {
                eventChan.close();
            } catch (IOException e) {
                throw new CloudRuntimeException(e);
            }
        }


        public void listen(Event evt, EventListenerWrapper l) {
            String type = evt.getType().toString();
            try {
                synchronized (listeners) {
                    List lst = listeners.get(type);
                    if (lst == null) {
                        lst = new CopyOnWriteArrayList();
                        listeners.put(type, lst);
                        eventChan.queueBind(queueName, BusExchange.BROADCAST.toString(), type);
                        logger.debug(String.format("[listening event]: %s", type));
                    }

                    if (!lst.contains(l)) {
                        lst.add(l);
                    }
                }
            } catch (IOException e) {
                throw new CloudRuntimeException(e);
            }
        }

        public void unlisten(Event evt, EventListenerWrapper l) {
            String type = evt.getType().toString();
            try {
                synchronized (listeners) {
                    List lst = listeners.get(type);
                    if (lst == null) {
                        return;
                    }

                    lst.remove(l);
                    if (lst.isEmpty()) {
                        listeners.remove(type);
                        eventChan.queueUnbind(queueName, BusExchange.BROADCAST.toString(), type);
                        logger.debug(String.format("[unlistening event]: %s", type));
                    }
                }
            } catch (IOException e) {
                throw new CloudRuntimeException(e);
            }
        }

        @SyncThread(level = 10)
        @MessageSafe
        private void dispatch(Event evt, EventListenerWrapper l) {
            setThreadLoggingContext(evt);

            l.callEventListener(evt);
        }


        private void handle(Event evt) {
            String type = evt.getType().toString();
            List lst = listeners.get(type);
            if (lst == null) {
                return;
            }

            if (logger.isTraceEnabled()) {
                logger.trace(String.format("[event received]: %s", wire.dumpMessage(evt)));
            }

            for (EventListenerWrapper l : lst) {
                dispatch(evt, l);
            }
        }

        @Override
        public void handleDelivery(String s, com.rabbitmq.client.Envelope envelope, AMQP.BasicProperties basicProperties, byte[] bytes) throws IOException {
            Event evt = null;
            try {
                evt = (Event) wire.toMessage(bytes, basicProperties);
                handle(evt);
            } catch (final Throwable t) {
                final Event fevt = evt;
                throwableSafe(new Runnable() {
                    @Override
                    public void run() {
                        if (fevt != null) {
                            logger.warn(String.format("unhandled throwable when handling event[%s], dump: %s", fevt.getClass().getName(), wire.dumpMessage(fevt)), t);
                        } else {
                            logger.warn(String.format("unhandled throwable"), t);
                        }
                    }
                });
            }
        }
    }

这段代码得先从handleDelivery开始看：

        @Override
        public void handleDelivery(String s, com.rabbitmq.client.Envelope envelope, AMQP.BasicProperties basicProperties, byte[] bytes) throws IOException {
            Event evt = null;
            try {
                evt = (Event) wire.toMessage(bytes, basicProperties);
                handle(evt);
            } catch (final Throwable t) {
                final Event fevt = evt;
                throwableSafe(new Runnable() {
                    @Override
                    public void run() {
                        if (fevt != null) {
                            logger.warn(String.format("unhandled throwable when handling event[%s], dump: %s", fevt.getClass().getName(), wire.dumpMessage(fevt)), t);
                        } else {
                            logger.warn(String.format("unhandled throwable"), t);
                        }
                    }
                });
            }
        }

可以看到，这里是重载了Consumer接口的handleDelivery，我们看一下它的方法注释：

   /**
     * Called when a basic.deliver is received for this consumer.
     * @param consumerTag the consumer tag associated with the consumer
     * @param envelope packaging data for the message
     * @param properties content header data for the message
     * @param body the message body (opaque, client-specific byte array)
     * @throws IOException if the consumer encounters an I/O error while processing the message
     * @see Envelope
     */
    void handleDelivery(String consumerTag,
                        Envelope envelope,
                        AMQP.BasicProperties properties,
                        byte[] body)
        throws IOException;

这样保证EventMaid的对象能够接收到Msg。在try代码块中，从byte转换出了Event，然后走向了handle逻辑。

        private void handle(Event evt) {
            //前面提过，有两种Type，API和Local
            String type = evt.getType().toString();
            //所以只会取出两种List
            List lst = listeners.get(type);
            if (lst == null) {
                return;
            }

            if (logger.isTraceEnabled()) {
                logger.trace(String.format("[event received]: %s", wire.dumpMessage(evt)));
            }

            for (EventListenerWrapper l : lst) {
                //跳到下一个逻辑
                dispatch(evt, l);
            }
        }

        @SyncThread(level = 10)
        @MessageSafe
        private void dispatch(Event evt, EventListenerWrapper l) {
            setThreadLoggingContext(evt);
            //跳至下一段逻辑
            l.callEventListener(evt);
        }

    @Override
    public EventSubscriberReceipt subscribeEvent(final CloudBusEventListener listener, final Event... events) {
        final EventListenerWrapper wrapper = new EventListenerWrapper() {
            @Override
            public void callEventListener(Event e) {
                //走到各自的handle逻辑，如果返回true则unlisten
                if (listener.handleEvent(e)) {
                    maid.unlisten(e, this);
                }
            }
        };
        // 一个event对应一个ListenWrapper
        for (Event e : events) {
            maid.listen(e, wrapper);
        }

        return new EventSubscriberReceipt() {
            @Override
            public void unsubscribe(Event e) {
                maid.unlisten(e, wrapper);
            }

            @Override
            public void unsubscribeAll() {
                for (Event e : events) {
                    maid.unlisten(e, wrapper);
                }
            }
        };
    }

再看listen:

        public void listen(Event evt, EventListenerWrapper l) {
            String type = evt.getType().toString();
            try {
                synchronized (listeners) {
                    List lst = listeners.get(type);
                    if (lst == null) {
                        lst = new CopyOnWriteArrayList();
                        listeners.put(type, lst);
                        eventChan.queueBind(queueName, BusExchange.BROADCAST.toString(), type);
                        logger.debug(String.format("[listening event]: %s", type));
                    }

                    if (!lst.contains(l)) {
                        lst.add(l);
                    }
                }
            } catch (IOException e) {
                throw new CloudRuntimeException(e);
            }
        }

首先加锁了listeners这个put，并根据type取出相应的list。同时将这个list转换为CopyOnWriteArrayList，这样这个list的引用就不会泄露出去了。然后绑定一个channel作为通道。另外，如果EventListenerWrapper List中不存在提交的EventListenerWrapper，则添加进去。

相信讲了这么多，有一部分读者可能已经绕晕了。这边写一个关于EventMaid的逻辑调用小结：

在ZStack的每个Component启动时，会向CloudBus订阅event。

当CloudBus收到需要publish的event，会向所有实现CloudBusEventListener接口的对象发送事件，由他们自己选择是否处理这些事件。

CloudBus和EventFascade就是这样协同工作的。

在本文，我们一起浏览的ZStack中提供消息驱动特性组件的源码——显然，这两个组件的API非常好用，简洁明了。但在具体逻辑中有几个可以改进的点：

handleEvent返回boolean的判断为ture则取消listen，语义上不是很好理解

listen方法中的listeners可以用并发容器——ConcurrentHashMap代替，以增加吞吐量。

listeners的v完全可以用Set来代替。CopyOnWriteArrayList也可以用CopyOnWriteArraySet来代替。我们在listen方法中可以看到，如果lst不包含l，则add。这说明lst是不应该重复的。