Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support react-native #11

Open
ortutay opened this issue Sep 11, 2015 · 49 comments
Open

Support react-native #11

ortutay opened this issue Sep 11, 2015 · 49 comments

Comments

@ortutay
Copy link

ortutay commented Sep 11, 2015

I am going through the tutorial and I hit an error when I try to connect to the local websocket server from my React Native app. The error is:

Error: Buffer is not defined
 stack: 
  Function.<anonymous>                                         index.ios.bundle:64324
  Function.module.exports.varar [as expr]                      index.ios.bundle:58951
  Table.<anonymous>                                            index.ios.bundle:61785
  Table.RDBOp [as constructor]                                 index.ios.bundle:61791
  new                                                          index.ios.bundle:62148
  Function.<anonymous>                                         index.ios.bundle:64402
  Function.module.exports.aropt [as table]                     index.ios.bundle:58972
  React.createClass.componentDidMount                          index.ios.bundle:1493
  CallbackQueue.assign.notifyAll                               index.ios.bundle:6674
  ReactNativeReconcileTransaction.ON_DOM_READY_QUEUEING.close  index.ios.bundle:16780
  ReactNativeReconcileTransaction.Mixin.closeAll               index.ios.bundle:7214
  ReactNativeReconcileTransaction.Mixin.perform                index.ios.bundle:7155
  batchedMountComponentIntoNode                                index.ios.bundle:7516
  Object.ReactDefaultBatchingStrategy.batchedUpdates           index.ios.bundle:16560
  Object.batchedUpdates                                        index.ios.bundle:6444
  Object.ReactNativeMount.renderComponent                      index.ios.bundle:7600
 URL: undefined
 line: undefined
 message: Buffer is not defined

Some relevant code:

var ReactRethinkdb = require('react-rethinkdb');
var r = ReactRethinkdb.r;

//...

ReactRethinkdb.DefaultSession.connect({
  host: 'localhost',
  port: 8015,
  path: '/db',
  secure: false,
  db: 'test',
});

//...

    console.log(r.table('turtles'));
@mikemintz
Copy link
Owner

I actually just tried to do this yesterday. I should have posted something about it. Try running this:

$ npm install buffer --save

And then add this to the top of the first javascript file you run (like index.ios.js):

global.Buffer = global.Buffer || require('buffer').Buffer;

Also, I think you'll need to require('react-rethinkdb/dist/node') instead of require('react-rethinkdb') since it's not running in a browser environment.

However, after I did all that, I ran into errors about some unknown http module, and didn't have enough time to resolve that yet. Let me know if you run into the same thing, or something else first, or find a way around it.

@ortutay
Copy link
Author

ortutay commented Sep 12, 2015

That solves this issue, thanks.

I still wasn't able to get this to work in React Native though. It seems like rethinkdb requires a few node core modules ('net' and 'tls') which React Native doesn't provide. Haven't figured out how to get past that yet...

@mikemintz
Copy link
Owner

Oh I think I know how to deal with those. I can try making a react native
distribution that stubs net and tls modules, and http too if that helps.
I'll try to get that by tomorrow, thanks for helping to test this!
On Sep 11, 2015 6:06 PM, "ortutay" notifications@github.com wrote:

That solves this issue, thanks.

I still wasn't able to get this to work in React Native though. It seems
like rethinkdb requires a few node core modules ('net' and 'tls') which
React Native doesn't provide. Haven't figured out how to get past that
yet...


Reply to this email directly or view it on GitHub
#11 (comment)
.

@ortutay
Copy link
Author

ortutay commented Sep 12, 2015

Great, thanks!

@mikemintz
Copy link
Owner

Actually I think it might be because the nodejs websocket library won't
work in react native. So I think it makes sense to use the original require
statement for the browser version of react-rethinkdb, and get
window,WebSocket defined using this polyfill:
facebook/react-native#890

I'll try it tomorrow but if you want to try sooner you can see if that
polyfill works for you.
On Sep 11, 2015 7:16 PM, "ortutay" notifications@github.com wrote:

Great, thanks!


Reply to this email directly or view it on GitHub
#11 (comment)
.

@mikemintz
Copy link
Owner

Unfortunately, I can't get the latest version of react-native to work easily, since it looks like it requires a newer xcode, which isn't available for os x 10.9.

Can you see if you can get the react-native websocket polyfill to work? You'll know it's working if you add console.log('ws', window.WebSocket) and it says something other than "ws undefined" in the xcode log.

And if that's working, then in theory the browser version require('react-rethinkdb') is the one that should work in react-native, since it doesn't depend on anything fancy other than websockets.

But I think the Buffer issue is actually telling, because browsers don't natively support Buffer either. Somehow the webpack build for rethinkdb-websocket-client is properly polyfilling Buffer for browsers, but it's not working for react-native. I'll try to look into that, but it'll be hard to debug until I get set up with a newer version of os x.

If you use require('react-rethinkdb') and the Buffer workaround above, what errors do you get?

@mikemintz mikemintz changed the title Getting "Buffer is not defined" error Support react-native Sep 12, 2015
@mikemintz
Copy link
Owner

Okay I think I understand the issue. In addition to having to polyfill window.WebSocket, which is apparently supported via facebook/react-native#890 , we have to replicate webpack's shims that it offers via https://github.com/webpack/node-libs-browser

I'm not sure what the best practice way of doing that is for react-native, and it looks like there's an open issue for it here facebook/react-native#1871

The Buffer workaround above is good, since it's a global. But the missing modules will mean we have to override what require does, and I'm unfamiliar with how that's set up in react-native. If you want to try https://github.com/mjohnston/react-native-webpack-server that might just work, but that might require completely changing your build process.

@ortutay
Copy link
Author

ortutay commented Sep 12, 2015

Does react-rethinkdb depend on the rethinkdb package? rethinkdb uses net and tls modules, which I think are not available in React Native.

Logging out Buffer via the workaround you provided seems to work and show something that looks correct, but using the same approach for net and tls gives undefined in both cases. I'll take a look at the solutions you linked above to see if it solve the problem.

@mikemintz
Copy link
Owner

Yes, react-rethinkdb depends indirectly on rethinkdb. But if you use require('react-rethinkdb') instead of require('react-rethinkdb/dist/node') it will use a patched version that doesn't depend on net or tls. Which require are you using that gives you the net and tls errors?

The approach for Buffer works because you can override the Buffer global variable with plain javascript, but net and tls are modules that can only be overridden with the help of the build system.

I believe if you use require('react-rethinkdb'), you shouldn't get net or tls errors.

Also, if you want to simplify the problem, you can try using rethinkdb-websocket-client by itself, which is what react-rethinkdb depends on and is likely what's causing the issues. Just npm install rethinkdb-websocket-client --save and copy/paste the example in the README to the bottom of your index.ios.js file from https://github.com/mikemintz/rethinkdb-websocket-client#how-do-i-use-this

@mikemintz
Copy link
Owner

Currently on iOS, this is blocked on react-native ArrayBuffer support:

On android, this is blocked on react-native WebSocket support:

When ArrayBuffers and WebSockets are working in react-native, react-rethinkdb should work with:

$ npm install events buffer --save
global.Buffer = global.Buffer || require('buffer').Buffer;
var ReactRethinkdb = require('react-rethinkdb');

@ide
Copy link

ide commented Oct 7, 2015

RN Android WebSockets are now in master. I don't think anyone has really stress-tested them if you want to give it a shot.

@fungilation
Copy link

Not much to add other than using ReactDB as backend for React Native is exactly what I'm looking for. Hopefully the blockers get resolved!

@fungilation
Copy link

On iOS ArrayBuffer support in RN. Looks like it's been resolved and merged? facebook/react-native#4483

@mikemintz
Copy link
Owner

@fungilation Yes I think that's the case. I saw that merge and tested it on react-native for android but turns out it was only implemented for iOS. Has anyone been able to test since then with iOS?

@fungilation
Copy link

I'll report after I setup my app and test, I have Mac and xcode

@mikemintz
Copy link
Owner

Thanks! Let me know if you run into any issues, so far I've been able to work around everything other than "array buffer not supported on this platform"

@xaviramirezcom
Copy link

Hi, It seems like there is no web socket implementation:

sendArrayBufferImpl(): void {
    // TODO
    console.warn('Sending ArrayBuffers is not yet supported');
  }

nothing is sent to server

https://github.com/facebook/react-native/blob/master/Libraries/WebSocket/WebSocket.js

@mikemintz
Copy link
Owner

Interesting, I guess they implemented receiving ArrayBuffers but not sending.

I think the most practical way forward is to add support in rethinkdb-websocket-client and rethinkdb-websocket-server for base64 string websocket messages. It shouldn't replace binary as the default, but that would be great if it allows react-native to work. I'll look into that.

@mikemintz
Copy link
Owner

I added a base64 option to rethinkdb-websocket-client and rethinkdb-websocket-server, which works fine in the browser.

Unfortunately, react-native android seems to ignore WebSocket sub-protocols. I just reported it as facebook/react-native#5810. But when I hard-code both the client and server to use base64 instead of negotiating the protocol, it works!

The iOS code looks like it respects WebSocket sub-protocols, so it should work without hard-coding client and server to base64.

If anyone wants to try it on iOS with latest master from rethinkdb-websocket-client and rethinkdb-websocket-server, setting wsProtocols: ['base64'] in RethinkdbWebsocketClient.connect(), that'd be nice to know if it works there. I'll probably push out a version of react-rethinkdb tonight that exposes wsProtocols so the whole thing should work on iOS.

@mikemintz
Copy link
Owner

If you use the latest versions (react-rethinkdb 0.5.4 and rethinkdb-websocket-server 0.3.6), you should be able to use base64 websockets by specifying wsProtocols: ['base64'] in RethinkSession.connect().

@xaviercobain88 and @fungilation, could you see if this works for you with react-native for iOS?

@mikemintz
Copy link
Owner

I was able to test on react-native for iOS on a Mac today, and found similar missing functionality in react-native's WebSocket implementation (it doesn't expose the sub-protocols information), which I just reported at facebook/react-native#6137.

Ideally we should resolve the WebSocket issues in react-native instead of working around them here. But since they may be time consuming to get working there, I propose we create an unofficial flag here to get things working in the meantime. I'll add something like forceBase64WebSockets in react-rethinkdb, rethinkdb-websocket-client, and rethinkdb-websocket-server, so when it's enabled we completely disregard the negotiated sub-protocol.

@lifuzu
Copy link

lifuzu commented May 3, 2016

+1
Just got in. I believe it is really an excited feature for react native and rethinkdb!

@GeoffreyPlitt
Copy link

Any update? Is there a definite way to make this work in RN?

@GeoffreyPlitt
Copy link

GeoffreyPlitt commented Aug 5, 2016

@mikemintz I'm using React-Native=0.29, rethinkdb-websocket-server=0.6.0 and react-rethinkdb=0.6.0. I'm using wsProtocols: ['base64'] in the client. Do I need to do something similar in the server?

I'm getting "list" argument must be an Array of Buffers in WebSocket.ws.onmessage when it tries to do Buffer.concat

@mikemintz
Copy link
Owner

@GeoffreyPlitt is this resolved by your fix in mikemintz/rethinkdb-websocket-client#9 (comment) ?

@GeoffreyPlitt
Copy link

No, I didn't get things working yet. I've gotten past one error, but having another.

It's not clear to me from your messages above, did anyone get this working with RN or not? Can that code be posted so I can see how this works correctly, at least in theory?

@babakness
Copy link

What is the current documented issue with getting this to work with RN?

@GeoffreyPlitt
Copy link

Same as OP. "Buffer is not defined". There are a handful of other global modules that browsers have but ReactNative doesn't have.

@mikemintz
Copy link
Owner

mikemintz commented Aug 22, 2016

Sorry I missed some of these messages.

I was able to work around "Buffer is not defined" with

$ npm install buffer --save
global.Buffer = global.Buffer || require('buffer').Buffer;

Since the issues I reported to react-native regarding websocket protocols look like they still haven't been resolved, I was only able to get it to work by locally modifying both rethinkdb-websocket-client and rethinkdb-websocket-server to force base64 regardless of the protocol agreed upon.

See these commits for the relevant lines

Note, it's not enough to set wsProtocols: ['base64'] in client, since that just determines what is sent to the server during negotiation.

Ideally this would be fixed in react-native's implementation of websocket (refer to the github issues I reported in the comments above). But in the meantime, if you need to use this without local modification, feel free to submit a PR with a hidden flag like forceBase64WebSockets

@mikemintz
Copy link
Owner

Other than adding hard-coding true in the rethinkdb-websocket-server/client code, here's some code I believe I was able to get working with android back on Feb 7:

package.json

{
  "name": "reactnativeandroidtest3",
  "version": "0.0.1",
  "private": true,
  "scripts": {
    "start": "node node_modules/react-native/local-cli/cli.js start"
  },
  "dependencies": {
    "buffer": "^4.0.0",
    "events": "^1.1.0",
    "react-native": "^0.25.1",
    "rethinkdb-websocket-client": "^0.4.6"
  }
}

Bottom of index.android.js

global.Buffer = require('buffer').Buffer;

var RethinkdbWebsocketClient = require('rethinkdb-websocket-client');
var r = RethinkdbWebsocketClient.rethinkdb;

var options = {
  host: '12d1ffa.ngrok.com',
  port: 80,
  path: '/db',
  wsProtocols: ['base64'],
  secure: false,
  db: 'test',
};

console.log("yay0");
RethinkdbWebsocketClient.connect(options).then(function(conn) {
  console.log("yay1");
  var query = r.table('turtles');
  query.run(conn, function(err, cursor) {
    console.log("yay2");
    cursor.toArray(function(err, results) {
      console.log("yay3");
      console.log(results);
    });
  });
}, function(e) {console.log("boo", e)});

@babakness
Copy link

babakness commented Aug 25, 2016

Hey @mikemintz, thanks I'll play around with it this evening. This might be worth a look--FeatherJS within RN. They use Socket.io

http://docs.feathersjs.com/clients/feathers.html#usage-with-react-native

@babakness
Copy link

Made the suggest change to rethinkdb-websocket-client

npm run test

Uncaught TypeError: "list" argument must be an Array of Buffers

@mikemintz
Copy link
Owner

The unit tests for rethinkdb-websocket-client are written assuming that the backend sends binary frames, not base64. I wouldn't worry that it's not passing, the code changes are just a workaround to force base64 no matter what on both ends.

@babakness
Copy link

babakness commented Aug 25, 2016

Using the hacked version as implied above: I setup a control on the browser. It works on the browser only if i DON'T pass wsProtocols: ['base64']. If I do I get the same error as the unit test.

Does not work on Android. Server never sees request. No errors on console, just 'yay0' and that's it.

@babakness
Copy link

Think you could create a repository to work on this and post your working code?

Thanks.

@babakness
Copy link

I believe I have iOS working right now. Please share working prototype for Android. Almost there...

@babakness
Copy link

babakness commented Aug 26, 2016

Now working for me on both Android and iOS. 👍

Had a chance to play around with this project some more last night. My problem revolved around the way RN Android routes traffic.

I didn't see this documented over at React-Native anywhere but the Android implementation of React Native seems to meddle with traffic over localhost. It also works in a way that effectively ignores /etc/hosts

Reverse proxy/Ngrok isn't strictly needed however (helpful for device testing for sure). When using Genymotion one can just use a different IP to reach the local machine (ie, 192.168...) and it should work with Android since straight up "localhost" doesn't, at least not for me.

🍷
To me, working with Rethink directly client-side offers a nicer API than Meteor's minimongo. The whitelist validation method is more intuitive. I've not seen or have been able to use minimongo in RN, its very tied to a browser environment. It's interesting that Rethink's Horizon has an API closer to Mongo than Rethink.

--- EDIT ---

Come to think of it, the localhost issue makes sense now because Genymotion's virtualization would have its own localhost.

@fungilation
Copy link

Good to know on comparing with minimongo. I switched to react native now as
meteor was inadequate for mobile. Considering backend and Realm is the
forerunner so far
On Fri, Aug 26, 2016 at 9:51 AM Babak notifications@github.com wrote:

Now working for me on both Android and iOS. 👍

Had a chance to play around with this project some more last night. My
problem revolved around the way RN Android routes traffic.

I didn't see this documented over at React-Native anywhere but the Android
implementation of React Native seems to not like 'localhost' or
'127.0.0.1'. It also works in a way that effectively ignores /etc/hosts

Reverse proxy/Ngrok isn't strictly needed however (helpful for device
testing for sure). When using Genymotion one can just use a different IP to
reach the local machine (ie, 192.168...) and it should work with Android
since straight up "localhost" doesn't, at least not for me.

🍷
To me, working with Rethink directly client-side offers a nicer API than
Meteor's minimongo. The whitelist validation method is more intuitive. I've
not seen or have been able to use minimongo in RN, its very tied to a
browser environment. It's interesting that Rethink's Horizon has an API
closer to Mongo than Rethink.


You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
#11 (comment),
or mute the thread
https://github.com/notifications/unsubscribe-auth/ADoJShUKb51cKlvMYQf9X-FmktAxPoUwks5qjxl3gaJpZM4F8HcK
.

@babakness
Copy link

@fungilation Thanks for the heads up on Realm. Seems like a great solution for local storage.

@fungilation
Copy link

Ya. For centralised data store, a sync between Realm <-> Firebase would be
interesting
On Fri, Aug 26, 2016 at 1:28 PM Babak notifications@github.com wrote:

@fungilation https://github.com/fungilation Thanks for the heads up on
Realm. Seems like a great solution for local storage.


You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
#11 (comment),
or mute the thread
https://github.com/notifications/unsubscribe-auth/ADoJSl2-ptIRLYZ6lHD0NziHQsL-iPUAks5qj0xUgaJpZM4F8HcK
.

@babakness
Copy link

Pull request in 🤖

mikemintz/rethinkdb-websocket-client#14

This PR should be all that is needed for base64 websocket. rethinkdb-websocket-server doesn't need any changes as of RN 0.32 (and perhaps previous versions). I don't believe react-rethinkdb would need to be patched either.

@fungilation
Copy link

Nice!
On Sat, Aug 27, 2016 at 2:47 AM Babak notifications@github.com wrote:

Pull request in 🤖

mikemintz/rethinkdb-websocket-client#14
mikemintz/rethinkdb-websocket-client#14

This PR should be all that is needed for base64 websocket.
rethinkdb-websocket-server doesn't need any changes as of RN 0.32 (and
perhaps previous versions). I don't believe react-rethinkdb would need to
be patched either.


You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
#11 (comment),
or mute the thread
https://github.com/notifications/unsubscribe-auth/ADoJSuxqfhnhGpKyUcNSBjRf24dDhycsks5qkAeygaJpZM4F8HcK
.

@GeoffreyPlitt
Copy link

Sweet! It's been a week, can we merge this?

@mikemintz
Copy link
Owner

This was merged and published a week ago, so it should work on the latest version from npm.

@GeoffreyPlitt
Copy link

Doh, sorry, thought this was a PR, not an issue.

@mikemintz
Copy link
Owner

No worries. I think the latest library works, but you still need to use this (below) in your client code. If you get a chance to try it out too, let us know if there are any issues.

$ npm install buffer --save
global.Buffer = global.Buffer || require('buffer').Buffer;

@babakness
Copy link

babakness commented Sep 3, 2016

@mikemintz On my end its not yet published on npm.

npm install rethinkdb-websocket-client
vi node_modules/rethinkdb-websocket-client/dist/index.js

            ws.onerror = function (event) {
              emitter.emit('error', event);
            };

            ws.onmessage = function (event) {
              var data = event.data;
              if (typeof Blob !== 'undefined' && data instanceof Blob) {
                (0, _blobToBuffer2['default'])(data, function (err, buffer) {
                  if (err) {
                    throw err;
                  }
                  emitter.emit('data', buffer);
                });
              } else if (typeof data === 'string' && ws.protocol === 'base64') {
                emitter.emit('data', new Buffer(data, 'base64'));
              } else {
                emitter.emit('data', data);
              }
            };
          };

How does one listen for the error being emitted via emitter.emit('error', event);. In React Native, interrupted connections can throw an error that trace back to this statement.

Error handling with rethinkdb-websocket-client should work as this:

r.table('marvel').run(conn).then(function(cursor) {
    return cursor.toArray()
}).then(function(results) {
    // process the results
}).catch(function(err) {
    // process error
})

https://www.rethinkdb.com/api/javascript/run/

Haven't yet tested. I'll follow up. Otherwise disconnecting even briefly will cause the infamous "red screen of death".

These are two different origins for exceptions btw. The first is when ws raises an exception the second is when this patched driver does.

@mikemintz
Copy link
Owner

@babakness good catch, I don't know why the publish didn't work for 0.5.0. I just republished as 0.5.1 and the changes should be there now.

I'm not quite sure what you're asking about the error handling. I think the emitter.emit('error', event) code should just be making websocket errors propagate as if they happened on the fake TCP socket. Are you saying that you think ws will raise an unhandled exception in addition to calling the onerror callback?

@babakness
Copy link

@mikemintz quick update, in RN, console.warn gives an annoying toast when it should just be something in the debug console. Also, for catching the rejected connection

https://github.com/babakness/react-rethinkdb/blob/master/src/Session.js#L43
https://github.com/babakness/react-rethinkdb/blob/master/src/Session.js#L60

Without this, the uncaught reject can trigger the "red screen of death" in RN.

I'm using a modified version of this Session functionality with Redux in RN. When a connection is lost, which is a given assumption in a mobile setting, we want to delegate the adjustment to this situation to the app codebase. To accommodate this need, an additional parameter was added--a callback function that can be passed in to trigger notification of this.

While the other functionality is nice--for example the feature that reduces redundant queries--this can be delegated out to the Redux workflow so that all related queries are handled in the same place.

I've sent a PR to review.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

8 participants