Email list creates but subscribers are never stored

[chiefoleka]

There seems to be a problem with the adding subscribers to the list.
Email list gets created (I can have like 1 million of them), but the subscribers never quite get added to the list. This happens using docker or local setup.

[karuppiah7890]

@chiefoleka Can you show a sample of your list including the header row ?

[chiefoleka]

I tried different combinations

email
email@gmail.com

Then I also tried

email,first name
email@gmail.com, firstname

Then also flipped this

first name, email
firstname, email@gmail.com

So, the thing is that the list comes in, it sees the headers and identifies each field irrespective of the combination I use, but it doesn't store it.

[karuppiah7890]

@chiefoleka Okay, I think I know what the issue is. I have encountered it, but I forgot to create an issue for this.

How big is your list ? Try uploading a subset of your list with say 500-1000 emails. And let me know if it works :)

I have encountered issues in uploading when the list was big - anything more than 2000 records (only emails). The upload didn't happen properly. It's some bug regarding uploading.

And the output that you see for the identification (the sample), it's done on the client side I think, and yes, looks like your list is in the right format. It just needs to have at least one column with the name "email". Even "Email" works I guess ? cc @AndrewGHC

[chiefoleka]

@karuppiah7890
Not necessarily. I tested with a single email because I felt it was the volume, but it still didn't work.
Now, it says upload completed but records 0.
The first time I set it up, I'm not sure, but I think it uploaded just fine, and I was running it in a docker container.
But before trying to use it last night, I sort of rebased my local repo people running docker. This issue started after then

[karuppiah7890]

@chiefoleka Then you could just stop and remove all containers using docker-compose down and try again if you don't have any important data in the DB.

@AndrewGHC I think we need to work on DB backup procedure (DB container) docs for people and provide volume based storage for DB in the upcoming versions so that it's easy to upgrade to the latest version of the application. And if for any reason we change DB schema and containers don't work in a plug and play manner, we can think on the DB upgrading procedures too later.

[chiefoleka]

@karuppiah7890
I'm not sure that would work because today, I removed all my docker containers since they are not necessarily useful yet.
But let me try and give you feedback

[chiefoleka]

@karuppiah7890
if it helps, here is the error that comes up every time I drop and recreate a container

app_1       | (node:51) DeprecationWarning: Using the automatically created return value from client.query as an event emitter is deprecated and will be removed in pg@7.0. Please see the upgrade guide at https://node-postgres.com/guides/upgrading
postgres_1  | ERROR:  relation "users" does not exist at character 112
postgres_1  | STATEMENT:  SELECT "id", "googleId", "picture", "token", "email", "name", "sentEmailsCount", "createdAt", "updatedAt" FROM "users" AS "user";

[karuppiah7890]

That's just an error when there are no users in the DB and when the App tries to run that statement to check if the users, before logging in. I see that too in a new container. But it will work once you login

@AndrewGHC I think we could handle this error this in a better manner ?

[karuppiah7890]

@chiefoleka If you still run into issues, ping me on gitter @karuppiah7890 and I will remote login to your PC through TeamViewer and see what the issue is :)

[karuppiah7890]

@chiefoleka Any updates on the issue ?

[chiefoleka]

Same issue I have always been having. It has not Vern fixed. Everything works, but once I get to the point where I have add list, it doesn't work.

…

On Thu, Sep 14, 2017, 3:19 PM Karuppiah ***@***.***> wrote: @chiefoleka <https://github.com/chiefoleka> Any updates on the issue ? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#189 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AKttHtP8Fel09SZS21NgiJi1ri4YHzREks5siTXegaJpZM4PR9Nc> .

[karuppiah7890]

"Vern" ?
And have you tried getting rid of the containers and recreating them using the commands I mentioned ?

[karuppiah7890]

@chiefoleka If you are still getting issues, let me check and verify the issue through TeamViewer. Ping me on gitter.im and we can decide on a time to check the issue

[karuppiah7890]

My Gitter username is karuppiah7890 , the same as my GitHub one

[chiefoleka]

We would do this later.

…

On 14 Sep 2017, at 3:26 PM, Karuppiah ***@***.***> wrote: @chiefoleka <https://github.com/chiefoleka> If you are still getting issues, let me check and verify the issue through TeamViewer. Ping me on gitter.im and we can decide on a time to check the issue — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#189 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AKttHuYNbTAbRquDmSHzO9NC_TjsdY-Dks5siTeogaJpZM4PR9Nc>.

[bmorelli25]

I think I figured out what the problem is. I too was having 0 emails come through, no matter my list size (tried 0, 100, and 1000 placeholder emails).

The fix, was renaming my email column. The email column must be named exactly email. I had it named Email with a capital E. Once I lowercased everything, my subscribers uploaded for all test sizes (0, 100, and 1000).

[bmorelli25]

@chiefoleka - make sure your header is titled exactly email. You may have an inadvertent character, capitalization, or space that's causing the issue. I'd delete the whole header row and add a new one just to be sure.

[karuppiah7890]

@bmorelli25 Thanks for that 😄 I think I have tried with Email as header. Let me try again. That's probably the issue as @chiefoleka couldn't import a list with even one mail id

[deltagolf]

As @bmorelli25 has said, I solved this issue by changing the header to "email" instead of "Email".

[deltagolf]

I've been searching the code and I think changing this line would fix this:

mail-for-good/server/controllers/list/import-csv.js

Line 35 in d1983c3

const additionalFields = _.without(JSON.parse(req.body.headers), 'email'); // e.g. name, location, sex, (excluding email header)

[pickleshy]

I too solved the issue by lowercasing "email" – would be very helpful if the helptext could be explicit about capitalization.