[GitHub] zeppelin pull request #2516: [ZEPPELIN-2827] Spark's pyspark fails if unsupp...

classic Classic list List threaded Threaded
7 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

[GitHub] zeppelin pull request #2516: [ZEPPELIN-2827] Spark's pyspark fails if unsupp...

asfgit
GitHub user prabhjyotsingh opened a pull request:

    https://github.com/apache/zeppelin/pull/2516

    [ZEPPELIN-2827] Spark's pyspark fails if unsupported version of matplotlib is present

    ### What is this PR for?
    %spark.pyspark paragraphs keep running for more than > 5 minutes and then at the end, throw TTransportException.
   
    ```
    org.apache.thrift.transport.TTransportException
    at org.apache.thrift.transport.TIOStreamTransport.read(TIOStreamTransport.java:132)
    at org.apache.thrift.transport.TTransport.readAll(TTransport.java:86)
    at org.apache.thrift.protocol.TBinaryProtocol.readAll(TBinaryProtocol.java:429)
    at org.apache.thrift.protocol.TBinaryProtocol.readI32(TBinaryProtocol.java:318)
    at org.apache.thrift.protocol.TBinaryProtocol.readMessageBegin(TBinaryProtocol.java:219)
    at org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:69)
    ```
   
    This happens because Zeppelin refers to `matplotlib.rcParams['savefig.format']` https://github.com/apache/zeppelin/blob/master/interpreter/lib/python/mpl_config.py#L83 which is only present in matplotlib 1.2.0 (https://github.com/matplotlib/matplotlib/blob/v1.2.0/matplotlibrc.template#L355) and onwards.
   
   
    ### What type of PR is it?
    [Bug Fix]
   
    ### What is the Jira issue?
    * [ZEPPELIN-2827](https://issues.apache.org/jira/browse/ZEPPELIN-2827)
   
    ### How should this be tested?
    Try running spark.pyspark on a machine with matplotlib==1.1.1 installed, pyspark should work as expected.
   
   
    ### Questions:
    * Does the licenses files need update? N/A
    * Is there breaking changes for older versions? N/A
    * Does this needs documentation? N/A


You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/prabhjyotsingh/zeppelin ZEPPELIN-2827

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/zeppelin/pull/2516.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #2516
   
----
commit da26f5613d831e74aacfcc08e91d4eed88e3d47c
Author: Prabhjyot Singh <[hidden email]>
Date:   2017-08-03T22:15:32Z

    add a version check along with check if matplotlib is installed

----


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [hidden email] or file a JIRA ticket
with INFRA.
---
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

[GitHub] zeppelin issue #2516: [ZEPPELIN-2827] Spark's pyspark fails if unsupported v...

asfgit
Github user prabhjyotsingh commented on the issue:

    https://github.com/apache/zeppelin/pull/2516
 
    @Leemoonsoo @zjffdu @agoodm please help review this.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [hidden email] or file a JIRA ticket
with INFRA.
---
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

[GitHub] zeppelin issue #2516: [ZEPPELIN-2827] Spark's pyspark fails if unsupported v...

asfgit
In reply to this post by asfgit
Github user Leemoonsoo commented on the issue:

    https://github.com/apache/zeppelin/pull/2516
 
    Looks good to me!


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [hidden email] or file a JIRA ticket
with INFRA.
---
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

[GitHub] zeppelin issue #2516: [ZEPPELIN-2827] Spark's pyspark fails if unsupported v...

asfgit
In reply to this post by asfgit
Github user prabhjyotsingh commented on the issue:

    https://github.com/apache/zeppelin/pull/2516
 
    Thank you, everyone, for the review. Will merge this if no more discussion.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [hidden email] or file a JIRA ticket
with INFRA.
---
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

[GitHub] zeppelin pull request #2516: [ZEPPELIN-2827] Spark's pyspark fails if unsupp...

asfgit
In reply to this post by asfgit
Github user prabhjyotsingh closed the pull request at:

    https://github.com/apache/zeppelin/pull/2516


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [hidden email] or file a JIRA ticket
with INFRA.
---
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

[GitHub] zeppelin pull request #2516: [ZEPPELIN-2827] Spark's pyspark fails if unsupp...

asfgit
In reply to this post by asfgit
GitHub user prabhjyotsingh reopened a pull request:

    https://github.com/apache/zeppelin/pull/2516

    [ZEPPELIN-2827] Spark's pyspark fails if unsupported version of matplotlib is present

    ### What is this PR for?
    %spark.pyspark paragraphs keep running for more than > 5 minutes and then at the end, throw TTransportException.
   
    ```
    org.apache.thrift.transport.TTransportException
    at org.apache.thrift.transport.TIOStreamTransport.read(TIOStreamTransport.java:132)
    at org.apache.thrift.transport.TTransport.readAll(TTransport.java:86)
    at org.apache.thrift.protocol.TBinaryProtocol.readAll(TBinaryProtocol.java:429)
    at org.apache.thrift.protocol.TBinaryProtocol.readI32(TBinaryProtocol.java:318)
    at org.apache.thrift.protocol.TBinaryProtocol.readMessageBegin(TBinaryProtocol.java:219)
    at org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:69)
    ```
   
    This happens because Zeppelin refers to `matplotlib.rcParams['savefig.format']` https://github.com/apache/zeppelin/blob/master/interpreter/lib/python/mpl_config.py#L83 which is only present in matplotlib 1.2.0 (https://github.com/matplotlib/matplotlib/blob/v1.2.0/matplotlibrc.template#L355) and onwards.
   
   
    ### What type of PR is it?
    [Bug Fix]
   
    ### What is the Jira issue?
    * [ZEPPELIN-2827](https://issues.apache.org/jira/browse/ZEPPELIN-2827)
   
    ### How should this be tested?
    Try running spark.pyspark on a machine with matplotlib==1.1.1 installed, pyspark should work as expected.
   
   
    ### Questions:
    * Does the licenses files need update? N/A
    * Is there breaking changes for older versions? N/A
    * Does this needs documentation? N/A


You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/prabhjyotsingh/zeppelin ZEPPELIN-2827

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/zeppelin/pull/2516.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #2516
   
----
commit da26f5613d831e74aacfcc08e91d4eed88e3d47c
Author: Prabhjyot Singh <[hidden email]>
Date:   2017-08-03T22:15:32Z

    add a version check along with check if matplotlib is installed

commit 0321fcd3e72382df8a97b1365b2194f9eb8632be
Author: Prabhjyot Singh <[hidden email]>
Date:   2017-08-04T05:48:40Z

    check if `savefig.format` exists in mpl_config.py

----


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [hidden email] or file a JIRA ticket
with INFRA.
---
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

[GitHub] zeppelin pull request #2516: [ZEPPELIN-2827] Spark's pyspark fails if unsupp...

asfgit
In reply to this post by asfgit
Github user asfgit closed the pull request at:

    https://github.com/apache/zeppelin/pull/2516


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [hidden email] or file a JIRA ticket
with INFRA.
---
Loading...