[GitHub] zeppelin pull request #2410: [ZEPPELIN-2645] Adding way to register RemoteIn...

classic Classic list List threaded Threaded
6 messages Options
Reply | Threaded
Open this post in threaded view
|

[GitHub] zeppelin pull request #2410: [ZEPPELIN-2645] Adding way to register RemoteIn...

prabhjyotsingh
GitHub user jongyoul opened a pull request:

    https://github.com/apache/zeppelin/pull/2410

    [ZEPPELIN-2645] Adding way to register RemoteInterpreterServer's port into InterpreterProcess

    ### What is this PR for?
    Changing the way to register interpreter to server. interpreter tells server its port of thrift server, and server connect interpreter through this information
   
    ### What type of PR is it?
    [Feature]
   
    ### Todos
    * [x] - And netty session for transferring data
   
    ### What is the Jira issue?
    * https://issues.apache.org/jira/browse/ZEPPELIN-2645
   
    ### How should this be tested?
    1. Set `SPARK_HOME` and `HADOOP_CONF_DIR`
    1. Set 'yarn-cluster' to spark's master
    1. Run a script
   
    ### Screenshots (if appropriate)
    ![image](https://user-images.githubusercontent.com/3612566/27132701-c9b3d708-514a-11e7-8892-f134d0423c84.png)
   
    ### Questions:
    * Does the licenses files need update? No
    * Is there breaking changes for older versions? No
    * Does this needs documentation? No


You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/jongyoul/zeppelin ZEPPELIN-2645

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/zeppelin/pull/2410.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #2410
   
----
commit 0570de5dc3c9f4be933bff18dd6fd2aa11536768
Author: Jongyoul Lee <[hidden email]>
Date:   2017-06-14T10:43:49Z

    Added getters

----


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [hidden email] or file a JIRA ticket
with INFRA.
---
Reply | Threaded
Open this post in threaded view
|

[GitHub] zeppelin issue #2410: [ZEPPELIN-2645] Adding way to register RemoteInterpret...

prabhjyotsingh
Github user zjffdu commented on the issue:

    https://github.com/apache/zeppelin/pull/2410
 
    @jongyoul I still don't understand why we can only set up only one server-client via netty. Maybe I miss something here.
    e.g. We set up netty server in zeppelin-server and netty client in interpreter process so that interpreter process can send request or push data to zeppelin server. But how does zeppelin server send request to interpreter process ? It seems we still need to set up another pair of netty server/client for that communication channel.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [hidden email] or file a JIRA ticket
with INFRA.
---
Reply | Threaded
Open this post in threaded view
|

[GitHub] zeppelin issue #2410: [ZEPPELIN-2645] Adding way to register RemoteInterpret...

prabhjyotsingh
In reply to this post by prabhjyotsingh
Github user jongyoul commented on the issue:

    https://github.com/apache/zeppelin/pull/2410
 
    @zjffdu Did you see the link I sent you? In that example, server keeps the channel when client bind at the first time. then, server can send any data through that channel when it wants to do it. I'll test it by my local laptop with two processes, and let you know


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [hidden email] or file a JIRA ticket
with INFRA.
---
Reply | Threaded
Open this post in threaded view
|

[GitHub] zeppelin issue #2410: [ZEPPELIN-2645] Adding way to register RemoteInterpret...

prabhjyotsingh
In reply to this post by prabhjyotsingh
Github user zjffdu commented on the issue:

    https://github.com/apache/zeppelin/pull/2410
 
    Yeah, I see that example. But it seems server can only send message to client only when it receive data from client (`SecureChatServerHandler.java`). How does server send request to client proactively ?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [hidden email] or file a JIRA ticket
with INFRA.
---
Reply | Threaded
Open this post in threaded view
|

[GitHub] zeppelin issue #2410: [ZEPPELIN-2645] Adding way to register RemoteInterpret...

prabhjyotsingh
In reply to this post by prabhjyotsingh
Github user jongyoul commented on the issue:

    https://github.com/apache/zeppelin/pull/2410
 
    Server broadcasts a message to all clients.
   
    On Tue, 27 Jun 2017 at 14:36 Jeff Zhang <[hidden email]> wrote:
   
    > Yeah, I see that example. But it seems server can only send message to
    > client only when it receive data from client (SecureChatServerHandler.java).
    > How does server send request to client proactively ?
    >
    > —
    > You are receiving this because you were mentioned.
    > Reply to this email directly, view it on GitHub
    > <https://github.com/apache/zeppelin/pull/2410#issuecomment-311257949>, or mute
    > the thread
    > <https://github.com/notifications/unsubscribe-auth/ADcflqQqO72PZMl3340rQPdyG4QYwkHvks5sIJTrgaJpZM4N51E2>
    > .
    >
    --
    이종열, Jongyoul Lee, 李宗烈
    http://madeng.net



---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [hidden email] or file a JIRA ticket
with INFRA.
---
Reply | Threaded
Open this post in threaded view
|

[GitHub] zeppelin issue #2410: [ZEPPELIN-2645] Adding way to register RemoteInterpret...

prabhjyotsingh
In reply to this post by prabhjyotsingh
Github user zjffdu commented on the issue:

    https://github.com/apache/zeppelin/pull/2410
 
    Yeah, I mean Server broadcasts a message to all clients only after it receive message from client. It looks like  Method `channelRead0` is invoked after it get message from client. There's no api to call this method directly.
   
    https://github.com/netty/netty/blob/4.0/example/src/main/java/io/netty/example/securechat/SecureChatServerHandler.java#L58


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [hidden email] or file a JIRA ticket
with INFRA.
---