Configure S3 End-Point for Delta Sharing

[yamenkaba]

Is there a way to configure an end-point for storage systems that support S3 API from Delta Sharing, as far as I know, delta sharing does not support hdfs and similar storage systems but there are some systems that provide S3 API support but configuring an end-point address is needed.

[zhuansunxt]

@yamenkaba as long as the storage system supports accessing data using a signed file path with limited time period (e.g. 1 hour), the Delta Sharing server should be able to support it. Do you have an example of such kind of storage system that supports S3 APIs?

[yamenkaba]

Yes, HPE Ezmeral Data Fabric it provide a S3 compatible API through its component that is called Object Store

[dgCunhaUA]

Hello,
I was able to connect delta sharing with Minio which is S3 compatible. All I needed to do was to create a core-site.xml inside /opt/docker/conf (using delta-sharing docker container) and specify the s3 endpoint like this:

<?xml version="1.0"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
<configuration>
 <property>
  <name>fs.s3a.endpoint</name>
  <value>http://ip:port</value>
 </property>
</configuration>

Dont know if that helps you or not but for minio worked.

[yamenkaba]

Thank you a lot, I will keep this in mind, but does the delta-sharing installed directly on the server(not docker) have a similar option, if yes how does it can be configured

[dgCunhaUA]

I believe that you can add that file to the /conf directory of the delta-sharing-server.