[BUG]: Unable to Register on Oppia.org #19858

dhruvmistry96 · 2024-03-01T20:56:26Z

Describe the bug

After signing in with Google and entering various types of Username I click on submit. But it does not proceed further.
The console log error is attached below.
Oppia Console Log error.log

I was able to replicate this issue on both Google Chrome and Microsoft Edge.

URL of the page where the issue is observed.

https://www.oppia.org/signup?return_url=%2F

Steps To Reproduce

Here is the video link to replicate this issue with Console Log open.

https://www.loom.com/share/7090d336c22a42f7aaca4f4838a368aa?sid=8fc0c40b-6982-4b04-8da7-49c3d1ef78a8

Expected Behavior

I expect to be directed to the next page and have either an error shown or successful registration.

Screenshots/Videos

https://www.loom.com/share/7090d336c22a42f7aaca4f4838a368aa?sid=8fc0c40b-6982-4b04-8da7-49c3d1ef78a8

What device are you using?

Desktop

Operating System

Windows

What browsers are you seeing the problem on?

Chrome

Browser version

122.0.6261.69 (Official Build) (64-bit)

Additional context

No response

Tips for developers

Before addressing the bug, please identify which PR caused the issue (you can follow the steps here). If you identify the PR, comment on the issue with a link to it. If not, mention the commit hash of the oldest commit you saw the bug on (and the month and year it was made in).

Also, if this is your first issue, please make sure to follow https://github.com/oppia/oppia/wiki/Contributing-code-to-Oppia#choosing-a-good-first-issue and https://github.com/oppia/oppia/wiki/Contributing-code-to-Oppia#setting-things-up before claiming it. Thanks!

HardikGoyal2003 · 2024-03-02T12:20:36Z

Closing this as a duplicate of #19837. Thanks!

seanlip · 2024-03-03T16:01:07Z

It might be related, but this doesn't seem like the same error exactly as #19837 (the other issue seems to have learner groups in the mix somewhere). So let's keep this open until we're sure.

dhruvmistry96 · 2024-03-04T03:43:51Z

While troubleshooting this, I found that selecting "Do not send these emails" for Email Preferences allows me to register successfully.

seanlip · 2024-03-04T15:08:28Z

@nikitaevg Could this be related? https://github.com/oppia/oppia/pull/19534/files

FYI it seems like the 500 error that is preventing signup-with-email only started appearning in the 3.3.5 release. The error is:

 Traceback (most recent call last):
  File "/layers/google.python.pip/pip/lib/python3.8/site-packages/webapp2.py", line 604, in dispatch
    return method(*args, **kwargs)
  File "/workspace/core/controllers/acl_decorators.py", line 2823, in test_login
    return handler(self, **kwargs)
  File "/workspace/core/controllers/profile.py", line 533, in post
    user_services.update_email_preferences(
  File "/workspace/core/domain/user_services.py", line 1665, in update_email_preferences
    bulk_email_services.add_or_update_user_status(
  File "/workspace/core/platform/bulk_email/mailchimp_bulk_email_services.py", line 299, in add_or_update_user_status
    user_creation_successful = _create_user_in_mailchimp_db(
  File "/workspace/core/platform/bulk_email/mailchimp_bulk_email_services.py", line 127, in _create_user_in_mailchimp_db
    raise Exception(error_message['detail']) from error
Exception: Your merge fields were invalid.

There is also another error which is still occurring but I'm not sure if it's related:

 Traceback (most recent call last):
  File "/workspace/core/controllers/base.py", line 358, in dispatch
    self.validate_and_normalize_args()
  File "/workspace/core/controllers/base.py", line 526, in validate_and_normalize_args
    raise self.InvalidInputException('\n'.join(errors))
core.controllers.base.UserFacingExceptions.InvalidInputException: Found extra args: ['data[merges][NAME]', 'data[merges][GROUPINGS][1][unique_id]', 'fired_at', 'data[merges][GROUPINGS][0][id]', 'data[web_id]', 'data[merges][REF]', 'data[email_type]', 'data[ip_opt]', 'data[merges][EMAIL]', 'data[id]', 'data[merges][INTERESTS]', 'data[merges][FNAME]', 'data[campaign_id]', 'data[reason]', 'data[merges][GROUPINGS][1][groups]', 'data[action]', 'data[merges][GROUPINGS][0][unique_id]', 'data[merges][GROUPINGS][0][groups]', 'data[merges][GROUPINGS][1][id]', 'data[merges][GROUPINGS][1][name]', 'data[merges][LNAME]', 'data[merges][GROUPINGS][0][name]'].

seanlip · 2024-03-04T17:02:12Z

Actually, @nikitaevg, scratch that, I think this isn't related to your changes.

It seems that the marketing team reconfigured the signup form and that's resulting in an error. I'm going to try and work with them to resolve this.

seanlip · 2024-03-06T15:52:42Z

For now, let's do the following fix:

If the mailchimp dependency fails during signup, we should log an error in Python.
However, we should still handle this properly in the frontend so that the user completes the rest of this signup and proceeds to the post-signup page. On that page we should show an alert message saying "Sorry, there was a problem and we were not able to sign you up for news and updates. Please sign-up directly using our newsletter here: https://shorturl.at/CHPY6."

U8NWXD · 2024-03-31T15:21:23Z

It looks like we already have code designed to handle mailchimp failures. If signup fails, we show a message like this on the signup page:

(To get this screenshot, I had to hard-code the signup URL and modify some checks in the backend because we don't normally show the email signup form or this error message when running locally.)

Here's the relevant frontend code:

oppia/core/templates/pages/signup-page/signup-page.component.html

Lines 72 to 83 in 8a4fda4

    
             <div class="alert alert-warning" *ngIf="showEmailSignupLink"> 
        
               <span class="help-block oppia-form-text" 
        
                     translate="I18N_PREFERENCES_EMAIL_SIGNUP_TEXT"> 
        
               </span> 
        
               <a class="help-block oppia-form-text" 
        
                  [href]="emailSignupLink" 
        
                  target="_blank" 
        
                  rel="noopener"> 
        
                 {{ emailSignupLink }} 
        
               </a> 
        
             </div> 
        
           </div>

And here's where we trigger the display of the error message in the controller layer:

oppia/core/controllers/profile.py

Lines 542 to 556 in 8a4fda4

    
           bulk_email_signup_message_should_be_shown = ( 
        
               user_services.update_email_preferences( 
        
                   self.user_id, can_receive_email_updates, 
        
                   feconf.DEFAULT_EDITOR_ROLE_EMAIL_PREFERENCE, 
        
                   feconf.DEFAULT_FEEDBACK_MESSAGE_EMAIL_PREFERENCE, 
        
                   feconf.DEFAULT_SUBSCRIPTION_EMAIL_PREFERENCE 
        
               ) 
        
           ) 
        
           if bulk_email_signup_message_should_be_shown: 
        
               self.render_json({ 
        
                   'bulk_email_signup_message_should_be_shown': ( 
        
                       bulk_email_signup_message_should_be_shown 
        
                   ) 
        
               }) 
        
               return

We already have error-handling code in core/platform/bulk_email/mailchimp_bulk_email_services.py (where the error causing this issue gets raised) that triggers the error message by returning False if the user doesn't exist:

oppia/core/platform/bulk_email/mailchimp_bulk_email_services.py

Lines 287 to 304 in 8a4fda4

    
           except mailchimpclient.MailChimpError as error: 
        
               # This has to be done since the message can only be accessed from 
        
               # MailChimpError by error.message in Python2, but this is deprecated in 
        
               # Python3. 
        
               # In Python3, the message can be accessed directly by KeyError 
        
               # (https://github.com/VingtCinq/python-mailchimp/pull/65), so as a 
        
               # workaround for Python2, the 'message' attribute is obtained by 
        
               # str() and then it is converted to dict. This works in Python3 as well. 
        
               error_message = ast.literal_eval(str(error)) 
        
               # Error 404 corresponds to 'User does not exist'. 
        
               if error_message['status'] == 404: 
        
                   if can_receive_email_updates: 
        
                       user_creation_successful = _create_user_in_mailchimp_db( 
        
                           client, new_user_mailchimp_data) 
        
                       if not user_creation_successful: 
        
                           return False 
        
               else: 
        
                   raise Exception(error_message['detail']) from error

I think we can solve this issue by expanding the error-handling code to also cover the Your merge fields were invalid. error and return False in this case as well after logging the error. @seanlip this isn't quite the solution you proposed (the error message appears on the signup page, and the user has to choose "Do not send these emails" to proceed). However, I think this approach is actually clearer for the user (there's no chance of the user missing the error message and then wondering why they aren't getting emails), and it makes better use of existing code. WDYT?

seanlip · 2024-03-31T18:14:22Z

@U8NWXD Yup this sounds fine, thanks! Let's just make sure a server error gets logged as well so that we can track whether this issue happens.

One thought -- maybe we should cover all exceptions, not just the specific "Your merge fields were invalid"? Basically, we want the user to still be able to sign up regardless of whether Mailchimp works or not.

U8NWXD · 2024-03-31T21:36:33Z

I have a fix implemented in https://github.com/U8NWXD/oppia/tree/fix-19858. Here's the flow:

flow.mov

Details on how I got the above recording and further testing of my approach

To simulate mailchimp failing and get debugging information, I applied this diff:

diff --git a/assets/constants.ts b/assets/constants.ts
index 8e3d5e8684..12824d08ff 100644
--- a/assets/constants.ts
+++ b/assets/constants.ts
@@ -6162,7 +6162,7 @@ export default {
 
   "MIN_QUESTION_COUNT_FOR_A_DIAGNOSTIC_TEST_SKILL": 3,
 
-  "BULK_EMAIL_SERVICE_SIGNUP_URL": "",
+  "BULK_EMAIL_SERVICE_SIGNUP_URL": "emailSignup.example.com",
 
   // The default number of opportunities to show on the contributor dashboard
   // page.
diff --git a/core/controllers/profile.py b/core/controllers/profile.py
index 7800cbc0bb..9ec8a673d6 100644
--- a/core/controllers/profile.py
+++ b/core/controllers/profile.py
@@ -530,6 +530,7 @@ class SignupHandler(
     @acl_decorators.require_user_id_else_redirect_to_homepage
     def post(self) -> None:
         """Handles POST requests."""
+        logging.error('[DEBUGGING] CAN_SEND_EMAILS=%s', feconf.CAN_SEND_EMAILS)
         assert self.user_id is not None
         assert self.normalized_payload is not None
         username = self.normalized_payload['username']
@@ -545,10 +546,16 @@ class SignupHandler(
             feconf.DEFAULT_FEEDBACK_MESSAGE_EMAIL_PREFERENCE,
             feconf.DEFAULT_SUBSCRIPTION_EMAIL_PREFERENCE,
         )
+        logging.error(
+            '[DEBUGGING] bulk_signup_failed=%s',
+            bulk_email_signup_message_should_be_shown)
         # Only block registration if bulk email configuration failed and the
         # user requested bulk emails.
         bulk_email_signup_message_should_be_shown = (
             can_receive_email_updates and config_bulk_email_failed)
+        logging.error(
+            '[DEBUGGING] bulk_email_signup_message_should_be_shown=%s',
+            bulk_email_signup_message_should_be_shown)
         if bulk_email_signup_message_should_be_shown:
             self.render_json({
                 'bulk_email_signup_message_should_be_shown': (
diff --git a/core/domain/user_services.py b/core/domain/user_services.py
index e1c01b39d5..d50c2b2cbd 100644
--- a/core/domain/user_services.py
+++ b/core/domain/user_services.py
@@ -1644,6 +1644,7 @@ def update_email_preferences(
     Returns:
         bool. Whether updating the user's bulk email preferences failed.
     """
+    logging.error('[DEBUGGING] update_email_preferences() called')
     email_preferences_model = user_models.UserEmailPreferencesModel.get(
         user_id, strict=False)
     if email_preferences_model is None:
@@ -1664,6 +1665,9 @@ def update_email_preferences(
             bulk_email_services.add_or_update_user_status(
                 email, {}, 'Account',
                 can_receive_email_updates=can_receive_email_updates))
+        logging.error(
+            '[DEBUGGING] user_creation_successful=%s',
+            user_creation_successful)
         if not user_creation_successful:
             email_preferences_model.site_updates = False
             email_preferences_model.update_timestamps()
diff --git a/core/feconf.py b/core/feconf.py
index 87faad4ce1..65a214d198 100644
--- a/core/feconf.py
+++ b/core/feconf.py
@@ -587,7 +587,7 @@ NOREPLY_EMAIL_ADDRESS = 'noreply@example.com'
 # correspond to owners of the app before setting this to True. If
 # SYSTEM_EMAIL_ADDRESS is not that of an app owner, email messages from this
 # address cannot be sent. If True then emails can be sent to any user.
-CAN_SEND_EMAILS = False
+CAN_SEND_EMAILS = True
 # If you want to turn on this facility please check the email templates in the
 # send_role_notification_email() function in email_manager.py and modify them
 # accordingly.
diff --git a/core/platform/bulk_email/dev_mode_bulk_email_services.py b/core/platform/bulk_email/dev_mode_bulk_email_services.py
index 552e3f1e8b..3654a6605b 100644
--- a/core/platform/bulk_email/dev_mode_bulk_email_services.py
+++ b/core/platform/bulk_email/dev_mode_bulk_email_services.py
@@ -61,4 +61,5 @@ def add_or_update_user_status(
         'Updated status of email ID %s\'s bulk email preference in the service '
         'provider\'s db to %s. Cannot access API, since this is a dev '
         'environment.' % (user_email, can_receive_email_updates))
-    return True
+    logging.error('[DEBUGGING] Simulating mailchimp failure.')
+    return False

The devserver logs confirm that the simulated failure was correctly interpreted by the backend code:

2024-03-31 17:15:14 ERROR:root:[DEBUGGING] CAN_SEND_EMAILS=True
2024-03-31 17:15:14 ERROR:root:[DEBUGGING] update_email_preferences() called
2024-03-31 17:15:14 ERROR:root:[DEBUGGING] Simulating mailchimp failure.
2024-03-31 17:15:14 ERROR:root:[DEBUGGING] user_creation_successful=False
2024-03-31 17:15:14 ERROR:root:[DEBUGGING] bulk_signup_failed=False
2024-03-31 17:15:14 ERROR:root:[DEBUGGING] bulk_email_signup_message_should_be_shown=True
2024-03-31 17:15:14 INFO     2024-03-31 21:15:14,914 module.py:883] default: "POST /signuphandler/data HTTP/1.1" 200 56
2024-03-31 17:15:15 INFO     2024-03-31 21:15:15,310 module.py:883] default: "GET / HTTP/1.1" 200 2310
2024-03-31 17:15:20 ERROR:root:[DEBUGGING] CAN_SEND_EMAILS=True
2024-03-31 17:15:20 ERROR:root:[DEBUGGING] update_email_preferences() called
2024-03-31 17:15:20 ERROR:root:[DEBUGGING] Simulating mailchimp failure.
2024-03-31 17:15:20 ERROR:root:[DEBUGGING] user_creation_successful=False
2024-03-31 17:15:20 ERROR:root:[DEBUGGING] bulk_signup_failed=False
2024-03-31 17:15:20 ERROR:root:[DEBUGGING] bulk_email_signup_message_should_be_shown=False
2024-03-31 17:15:20 ERROR:root:Please ensure that the value for the admin platform property SIGNUP_EMAIL_SUBJECT_CONTENT is set, before allowing post-signup emails to be sent.

Note that there's a mistake in the debugging code. The value logged for bulk_signup_failed is actually bulk_email_signup_message_should_be_shown, which you can see from the diff above.

I also tried with the browser console open:

flow_console_error.mov

The console errors also happen in develop:

before_flow.mov

Does this UX look good to you @seanlip? If so I'll write the tests and open a PR

seanlip · 2024-04-02T10:43:57Z

Thanks @U8NWXD, this looks great. Please go ahead with the PR. One note though: I would suggest treating that PR as a partial fix of this issue, we should still leave this open for the full fix (or perhaps file a new issue to track that).

Thanks!

U8NWXD · 2024-04-27T15:00:13Z

#20143 failed to fix the problem. Here is the new error log:

Exception raised at https://www.oppia.org/signuphandler/data: Your merge
fields were invalid.
Traceback (most recent call last):
  File
"/workspace/core/platform/bulk_email/mailchimp_bulk_email_services.py",
line 269, in add_or_update_user_status
    client.lists.members.get(
  File
"/layers/google.python.pip/pip/lib/python3.8/site-packages/mailchimp3/entities/listmembers.py",
line 118, in get
    return self._mc_client._get(url=self._build_path(list_id, 'members',
subscriber_hash), **queryparams)
  File
"/layers/google.python.pip/pip/lib/python3.8/site-packages/mailchimp3/mailchimpclient.py",
line 30, in wrapper
    return fn(self, *args, **kwargs)
  File
"/layers/google.python.pip/pip/lib/python3.8/site-packages/mailchimp3/mailchimpclient.py",
line 168, in _get
    _raise_response_error(r)
  File
"/layers/google.python.pip/pip/lib/python3.8/site-packages/mailchimp3/mailchimpclient.py",
line 44, in _raise_response_error
    raise MailChimpError(error_data)
mailchimp3.mailchimpclient.MailChimpError: {'type': '
https://mailchimp.com/developer/marketing/docs/errors/', 'title': 'Resource
Not Found', 'status': 404, 'detail': 'The requested resource could not be
found.', 'instance': '*some-base64-string*'}

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File
"/workspace/core/platform/bulk_email/mailchimp_bulk_email_services.py",
line 114, in _create_user_in_mailchimp_db
    client.lists.members.create(
  File
"/layers/google.python.pip/pip/lib/python3.8/site-packages/mailchimp3/entities/listmembers.py",
line 60, in create
    response = self._mc_client._post(url=self._build_path(list_id,
'members'), data=data)
  File
"/layers/google.python.pip/pip/lib/python3.8/site-packages/mailchimp3/mailchimpclient.py",
line 30, in wrapper
    return fn(self, *args, **kwargs)
  File
"/layers/google.python.pip/pip/lib/python3.8/site-packages/mailchimp3/mailchimpclient.py",
line 135, in _post
    _raise_response_error(r)
  File
"/layers/google.python.pip/pip/lib/python3.8/site-packages/mailchimp3/mailchimpclient.py",
line 44, in _raise_response_error
    raise MailChimpError(error_data)
mailchimp3.mailchimpclient.MailChimpError: {'type': '
https://mailchimp.com/developer/marketing/docs/errors/', 'title': 'Invalid
Resource', 'status': 400, 'detail': 'Your merge fields were invalid.',
'instance': '*some-base64-string*', 'errors': [{'field': 'FNAME',
'message': 'Please enter a value'}, {'field': 'LNAME', 'message': 'Please
enter a value'}]}

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File
"/layers/google.python.pip/pip/lib/python3.8/site-packages/webapp2.py",
line 604, in dispatch
    return method(*args, **kwargs)
  File "/workspace/core/controllers/acl_decorators.py", line 2823, in
test_login
    return handler(self, **kwargs)
  File "/workspace/core/controllers/profile.py", line 542, in post
    config_bulk_email_failed = user_services.update_email_preferences(
  File "/workspace/core/domain/user_services.py", line 1664, in
update_email_preferences
    bulk_email_services.add_or_update_user_status(
  File
"/workspace/core/platform/bulk_email/mailchimp_bulk_email_services.py",
line 298, in add_or_update_user_status
    user_creation_successful = _create_user_in_mailchimp_db(
  File
"/workspace/core/platform/bulk_email/mailchimp_bulk_email_services.py",
line 127, in _create_user_in_mailchimp_db
    raise Exception(error_message['detail']) from error
Exception: Your merge fields were invalid.

Note that a number of strings were redacted and replaced with *some-base64-string*

U8NWXD · 2024-04-27T15:08:39Z

Solutions:

Fix the underlying bug, which is likely because of first and last name fields not being provided to mailchimp
Fix the error handling with two try-except blocks. One to handle users not existing, and an outer one to handle all registration failures

U8NWXD · 2024-05-04T14:58:08Z

We confirmed that after making the first name and last name fields optional on Mailchimp, registration succeeded. Remaining TODOs:

Coordinate with marketing team to figure out whether the name fields should be required. Proposed solution: Make first name required and send username as first name when signing-up new users.
Still need error-handling to make sure mailchimp issues don't actually block registration.

* Make MailChimp Error Handling Comprehensive Ensure that all errors from MailChimp while signing a user up for the mailing list are caught, even those errors that arise when trying to handle missing users. * Fix linter

dhruvmistry96 added bug Label to indicate an issue is a regression triage needed labels Mar 1, 2024

HardikGoyal2003 closed this as not planned Won't fix, can't repro, duplicate, stale Mar 2, 2024

seanlip reopened this Mar 3, 2024

seanlip added Impact: High Blocks or significantly slows down a core workflow. Work: Medium The means to find the solution is clear, but it isn't at good-first-issue level yet. and removed triage needed labels Mar 4, 2024

hrshkshri mentioned this issue Mar 14, 2024

Create Account Button not Working #19935

Closed

seanlip mentioned this issue Mar 18, 2024

Difficulty signing into Oppia website. oppia/lessons-team#1099

Closed

U8NWXD self-assigned this Mar 30, 2024

U8NWXD mentioned this issue Apr 8, 2024

Fix part of 19858: Catch and Log Mailchimp Signup Errors #20143

Merged

5 tasks

U8NWXD mentioned this issue May 7, 2024

Fix #19858: Make MailChimp Error Handling Comprehensive #20272

Merged

5 tasks

seanlip closed this as completed in #20272 May 20, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[BUG]: Unable to Register on Oppia.org #19858

[BUG]: Unable to Register on Oppia.org #19858

dhruvmistry96 commented Mar 1, 2024 •

edited

HardikGoyal2003 commented Mar 2, 2024

seanlip commented Mar 3, 2024

dhruvmistry96 commented Mar 4, 2024

seanlip commented Mar 4, 2024 •

edited

seanlip commented Mar 4, 2024

seanlip commented Mar 6, 2024

U8NWXD commented Mar 31, 2024

seanlip commented Mar 31, 2024

U8NWXD commented Mar 31, 2024 •

edited

seanlip commented Apr 2, 2024

U8NWXD commented Apr 27, 2024

U8NWXD commented Apr 27, 2024

U8NWXD commented May 4, 2024

[BUG]: Unable to Register on Oppia.org #19858

[BUG]: Unable to Register on Oppia.org #19858

Comments

dhruvmistry96 commented Mar 1, 2024 • edited

Describe the bug

URL of the page where the issue is observed.

Steps To Reproduce

Expected Behavior

Screenshots/Videos

What device are you using?

Operating System

What browsers are you seeing the problem on?

Browser version

Additional context

Tips for developers

HardikGoyal2003 commented Mar 2, 2024

seanlip commented Mar 3, 2024

dhruvmistry96 commented Mar 4, 2024

seanlip commented Mar 4, 2024 • edited

seanlip commented Mar 4, 2024

seanlip commented Mar 6, 2024

U8NWXD commented Mar 31, 2024

seanlip commented Mar 31, 2024

U8NWXD commented Mar 31, 2024 • edited

seanlip commented Apr 2, 2024

U8NWXD commented Apr 27, 2024

U8NWXD commented Apr 27, 2024

U8NWXD commented May 4, 2024

dhruvmistry96 commented Mar 1, 2024 •

edited

seanlip commented Mar 4, 2024 •

edited

U8NWXD commented Mar 31, 2024 •

edited