[MINOR] Fix deployment issues

[jongyoul]

What is this PR for?

Fixing some issues that happened when releasing.

What type of PR is it?

Hot Fix

Todos

• - Remove some plugins from the shell interpreter instead of removing them from the distribution as some interpreters depend on it
• - Upgrade python version for docker image to 3.9

What is the Jira issue?

N/A

How should this be tested?

Distribution and building docker image should work fine.

Screenshots (if appropriate)

Questions:

• Does the license files need to update? No
• Is there breaking changes for older versions? No
• Does this needs documentation? No

[pan3793]

what's the effect of this change?

[jongyoul]

When publishing all interpreters including shell, maven-shade-plugin makes a fatjar and copies it into interpreter directory. maven-resource-plugin works similarly except for copying configurations to the directory.

FYI,

zeppelin/zeppelin-interpreter-parent/pom.xml

Lines 110 to 164 in f2253f1

	<!-- generate interpreter shade jar and put it under folder interpreter/${interpreter.name}-->
	<plugin>
	<groupId>org.apache.maven.plugins</groupId>
	<artifactId>maven-shade-plugin</artifactId>
	<configuration>
	<createDependencyReducedPom>false</createDependencyReducedPom>
	<filters>
	<filter>
	<artifact>*:*</artifact>
	<excludes>
	<exclude>META-INF/*.SF</exclude>
	<exclude>META-INF/*.DSA</exclude>
	<exclude>META-INF/*.RSA</exclude>
	<exclude>org.apache.zeppelin:zeppelin-interpreter-shaded</exclude>
	</excludes>
	</filter>
	</filters>
	<transformers>
	<transformer implementation="org.apache.maven.plugins.shade.resource.ServicesResourceTransformer" />
	<transformer implementation="org.apache.maven.plugins.shade.resource.AppendingTransformer">
	<resource>reference.conf</resource>
	</transformer>
	</transformers>
	<artifactSet>
	<excludes>
	<exclude>org.apache.zeppelin:zeppelin-interpreter-shaded</exclude>
	</excludes>
	</artifactSet>
	<outputFile>${project.basedir}/../interpreter/${interpreter.name}/${project.artifactId}-${project.version}.jar</outputFile>
	</configuration>
	<executions>
	<execution>
	<phase>package</phase>
	<goals>
	<goal>shade</goal>
	</goals>
	</execution>
	</executions>
	</plugin>
	<plugin>
	<artifactId>maven-resources-plugin</artifactId>
	<executions>
	<execution>
	<id>copy-interpreter-setting</id>
	<phase>package</phase>
	<goals>
	<goal>resources</goal>
	</goals>
	<configuration>
	<outputDirectory>${project.build.directory}/../../interpreter/${interpreter.name}</outputDirectory>
	</configuration>
	</execution>
	</executions>
	</plugin>

[pan3793]

It seems the whole sh dir disappears in the interpreter folder, not only "configurations", is it expected?

[jongyoul]

Yes, correct. Shell interpreter was and will be removed from the official binary because it lead a lot of security issues.

[pan3793]

LGTM if it's the expected behavior. Test may need to be adjusted to adapt this change.

[jongyoul]

There are a lot of test failures. Let me fix it one by one soon