(s2Member / API Notification Email) - Payment - wrong user data

Oh actually I just noticed a difference in my setup - I don’t know why - but for the website with working API emails - I have this line in the payment API notification URL - this doesn’t exist on the other website. Is this a setting that should exist and somehow went missing? It should not be that the value is needed for correct payment notification emails for my understanding…
So maybe the bug of users not demoted correctly and the payment emails is not related? Or is this line which was missing actually required? (if required then it should not be possible to delete it!).

s2member settings --> API Notifications --> Payment Notification URLs (optional) --> value:

https://www.velomap.org/s2_payment_notification=yes&user_id=%%user_id%%&amount=%%amount%%&currency_symbol=%%currency_symbol%%&txn_id=%%txn_id%%&currency=%%currency%%&payer_email=%%payer_email%%&item_name=%%item_name%%&item_number=%%item_number%%`

Note - the domain here is my own website - so I have no clue what is sent where here…

Nope - no change. I’m really lost - dunno what else to look at. Everything the same on my two websites - but on one the payment emails broken, on the other working.

Interesting. Really hard from me to tell what is causing the odd behavior looking at that. I don’t see something wrong, but I understand that you say that the user didn’t exist in the first one.

Right, it’s not a subscription, but it says web_accept (i.e. buy-now), so it’s only a wording thing but it’s handling it correctly.

The notification would not affect the other thing… I do notice that your URL is wrong, it’s missing the question mark before the querystring. Not sure if that’s only an error when posting it here but it’s fine on your configuration.

https://www.velomap.org/?s2_payment_notification=yes&user_id=%%user_id%%&amount=%%amount%%&currency_symbol=%%currency_symbol%%&txn_id=%%txn_id%%&currency=%%currency%%&payer_email=%%payer_email%%&item_name=%%item_name%%&item_number=%%item_number%%

All I can think of to troubleshoot that, is go back to the basics: install a clean WP, new s2, configure, make it work, then add a single thing at a time, testing after each.

I’d start by adding your customizations. If you can then make the odd behavior happen, you’ll know exactly what changed right before and you can investigate that specific thing without all the other variables.

:slight_smile:

Okay - will test that now. Let’s see. But i cannot be a plugin or a theme - tried on twenty nineteen with all other plugins off already. Could be some leftover in database of course.

1 Like

It does not help. I went back to the basics - started with a clean database/clean wp-contents folder. All works (also now s2Member / API Notification email is sent for test payments and not only for live payments).

Then rsynced the wp-content folder - and activated all plugins as on my broken page. Still everything working. Imported my theme settings, no problems. Imported s2member-settings - no problems.

Switch to broken database by changing wp-config.php database file --> Bang problems appear. I’m sure over the years s2member ended up putting some garbage into the s2member option in the database. Deactivating with safe-guards of, then reactivating does not help. Still broken. So I opened up a thread how to fully reset s2member - because deactivating clearly does not empty all the tables s2member uses in the database.

Way earlier @clavaque said this:

“or there’s some funky cache issue going on”

But I searched this super long thread for “cache” and “host” and didn’t see much discussed about this.

So…

Where are you hosting?

Are you using Cloudflare?

Nope, dedicated Ubuntu 18.04 root server. nginx 16.01 with php-fpm. No server wide caching. Not using cloudflare (except as DNS).

It’s certainly not Cache related

Strange, then.

Sorry that I haven’t experienced it to be able to help more. I do know that s2 (at least used to be) picky about caching and hosting. It would choke on Flywheel or WPengine.

Oh, I don’t know about now, but s2 really used to prefer Apache and .htaccess, but this concern is so dated and your Ubuntu version is so new that it’s probably not that. My last s2 site is on Kinsta now (nginx, magic host caching) and it’s working. (I was migrating everything until @clavaque came back and am keeping this last one to be able to participate).

As another user that’s all I got that I’ve somewhat experienced.

It’s clearly related to something in the database. And I have this problem since ages (other plugins and hosting changed a couple of times over this timeframe).

But this got me to an idea: one one of my websites (velomap.org) - the problem is solved since 03 April 2016 (still Stripe users not demoted - but then they were all created while the problem already existed.
First wrong email: 24 November 2012

On my other site - openmtbmap.org - Problem is still there. First occurence 03 January 2013.

Problem is of course - I don’t have any database backup that old to find out what I changed on those dates…
Both websites were always hosted the same way. So that the first time this happened is a different date - also shows it’s not server related.

okay - if I drop the wp_usermeta table - then everything works fine. Any idea how to find out where s2member chokes on the wp_usermeta table (of course it could also be on the wp_users table instead - as they are linked to each other I really cannot tell)?

It’s definitely not related to wp_options as I started from scratch with a fresh wp_options table - but problem was there. So I guess now that at some point due to corruption or a strange plugin this turned wrong. Likely for my second site I deleted a user that caused the problem so it started working again.

I checked the ID for the user it always inserts - it’s not the problem - also around that ID all entries look fine.

I have 23.000 users - so those tables are huge…
If I export all users, delete wp_usermeta and wp_users - then reimport the table - all passwords are lost if I’m not wrong. That would cause huge support work (because even If I send out emails telling people there password had to be reset - I will have to manually answer thousands of emails with users asking about the password and why it got reset - besides the trouble of sending out 23.000 emails). It’s not a viable solution if passwords are lost.

Okay I think I got an Idea how to solve this. Use the Wordpress Import/Export User tool to export all users (just ID and Password). Then Export all users with s2member. Merge all s2member exports into one file. Load both files into excel. Now query for ID and add the ID field to s2member exports. Now split the file again into pieces of 1000. Delete the wp_users and wp_usermeta table. Create an admin login. Now reimport all users.

It’s a PITA that s2member import/export tools cannot export passwords. The passwords are hashed but not related to anything else. So s2member should simply export the pass field too, and reimport it too. Also make it possible to have more than 1000 users at one go. If you properly setup php no problem to import/export 50.000 users in one pass.

Bummer: s2member cannot import password hashes. So above doesn’t work. The only way would be to do this as an sql query - but I have no idea how to run such a huge (150MB of data) query. So that’s not feasible either.

Basically I think I’m fu**ed - I cannot import/export the users, and at the same time don’t know how to find out why s2member has a problem with my wp_usermeta table (I already checked - there are no users that exist in wp_usermeta but don’t exist in wp_users or other way around).

Okay - so let’s document the progress:

  1. Try - Delete 10.000 users:
    DELETE FROM wp_users WHERE ID>=3 and ID < 10000;

DELETE FROM wp_usermeta
WHERE NOT EXISTS (
SELECT * FROM wp_users
WHERE wp_usermeta.user_id = wp_users.ID
)

(to give you an idea about the spead of the database:
269080 rows deleted. (Query took 10.1938 sec))

Problem solved (ID=2 is myself/admin so I cannot delete that one).

  1. Try - Restore Database Tables - then Delete the other half of the users (except my own account.
    DELETE FROM wp_users WHERE ID>=9999;
    12586 rows deleted. (Query took 0.1062 sec)

DELETE FROM wp_usermeta
WHERE NOT EXISTS (
SELECT * FROM wp_users
WHERE wp_usermeta.user_id = wp_users.ID
)
397767 rows deleted. (Query took 3.7037 sec)

Still not working. So there is clearly a problem with one or several of the first 10.000 users.

2b) Try to - drop all keys from wp_usermeta that I don’t need. Maybe that solves it already?

DELETE FROM wp_usermeta WHERE meta_key = ‘aim’;
DELETE FROM wp_usermeta WHERE meta_key = ‘rich_editing’;
DELETE FROM wp_usermeta WHERE meta_key = ‘comment_shortcuts’;
DELETE FROM wp_usermeta WHERE meta_key = ‘admin_color’;
DELETE FROM wp_usermeta WHERE meta_key = ‘closedpostboxes_dashboard’;
DELETE FROM wp_usermeta WHERE meta_key = ‘metaboxhidden_dashboard’;
DELETE FROM wp_usermeta WHERE meta_key = ‘screen_layout_dashboard’;
DELETE FROM wp_usermeta WHERE meta_key = ‘wp_dashboard_quick_press_last_post_id’;
DELETE FROM wp_usermeta WHERE meta_key = ‘managenav-menuscolumnshidden’;
DELETE FROM wp_usermeta WHERE meta_key = ‘metaboxhidden_nav-menus’;
DELETE FROM wp_usermeta WHERE meta_key = ‘closedpostboxes_page’;
DELETE FROM wp_usermeta WHERE meta_key = ‘metaboxhidden_page’;
DELETE FROM wp_usermeta WHERE meta_key = ‘meta-box-order_page’;
DELETE FROM wp_usermeta WHERE meta_key = ‘screen_layout_page’;
DELETE FROM wp_usermeta WHERE meta_key = ‘use_ssl’;
DELETE FROM wp_usermeta WHERE meta_key = ‘show_admin_bar_front’;
DELETE FROM wp_usermeta WHERE meta_key = ‘yim’;
DELETE FROM wp_usermeta WHERE meta_key = ‘jabber’;
DELETE FROM wp_usermeta WHERE meta_key = ‘locale’;
DELETE FROM wp_usermeta WHERE meta_key = ‘syntax_highlighting’;
DELETE FROM wp_usermeta WHERE meta_key = ‘default_password_nag’;
DELETE FROM wp_usermeta WHERE meta_key = ‘wikipedia’;
DELETE FROM wp_usermeta WHERE meta_key = ‘youtube’;
DELETE FROM wp_usermeta WHERE meta_key = ‘community-events-location’;
DELETE FROM wp_usermeta WHERE meta_key = ‘twitter’;
DELETE FROM wp_usermeta WHERE meta_key = ‘session_token’;
DELETE FROM wp_usermeta WHERE meta_key = ‘instagram’;
DELETE FROM wp_usermeta WHERE meta_key = ‘linkedin’;
DELETE FROM wp_usermeta WHERE meta_key = ‘myspace’;
DELETE FROM wp_usermeta WHERE meta_key = ‘pinterest’;
DELETE FROM wp_usermeta WHERE meta_key = ‘soundcloud’;
DELETE FROM wp_usermeta WHERE meta_key = ‘tumblr’;
DELETE FROM wp_usermeta WHERE meta_key = ‘facebook’;
DELETE FROM wp_usermeta WHERE meta_key LIKE ‘wpseo%’;
DELETE FROM wp_usermeta WHERE meta_key LIKE ‘_yoast%’;
DELETE FROM wp_usermeta WHERE meta_key = ‘description’;

Cleans up the database a bit - but still broken.

2c)
Try to delete the user that always shows up in the emails (will there be another new one then?).

Bingo works. So one of the keys on this user must be making s2member choke! Now let’s find out what key and which value! Will continue in next post

1 Like

Okay - so first remove the useless data from the usermeta_table, just for an indication - This decreases the size of user_meta table from
rows: ~631,550 -163.8 MiB to ~408,555 - 118.7 MiB - it’s not needed to solve anything here - but as I already cleaned up wp_options I don’t want to stop here (by cleaning up wp_options I brought my total database size down by 1/3 - there were millions transients left over plus more rubbish data leftover from plugins a long long time).

I had a typo above - deleting session tokens - so first:
DELETE FROM wp_usermeta WHERE meta_key = ‘session_tokens’;
(not such a good idea because everyone will have to login again). Does not help anyhow. So do not delete them.
It does clean up the database quite a lot however - down to: 361.500 rows and 104MB

  1. Delete: wp_s2member_paid_registration_times
    Nope - still broken:

  2. Delete: wp_s2member_ipn_signup_vars
    Nope - still broken:

  3. Delete: wp_s2member_file_download_access_arc
    and wp_s2member_file_download_access_log
    Still broken

  4. Delete: wp_s2member_first_payment_txn_id
    and wp_s2member_access_cap_times

  5. paid registration time:
    I give up. S2member behaves super strange!

So actually after deleting those keys - s2member will start writing new keys to the old user ID 288…

wp_s2member_paid_registration_times
wp_s2member_first_payment_txn_id
wp_s2member_ipn_signup_vars

They are written here - wp_s2member_paid_registration_times is written to both users. The other 2 keys are missing from the new user! And I guess those two keys missing explains why Stripe subscription users are not demoted? Or maybe on demotion the same thing happens?

Nice troubleshooting work!

So actually after deleting those keys - s2member will start writing new keys to the old user ID 288…

That’s the user that keeps coming up in your logs? That’s so weird… There must be something hardcoded putting that in…

It’s most likely not in the s2Member files, or it would have been overwritten by any of the updates.

That’s why I kept asking about cache or custom code… Something is putting that number there. And it’s not a random thing, if it’s always the same user.

Can you search all your files, and all your database, for “288” and see what you get?

Okay - I found the error - I was a bit blind to not see this earlier - but here is the ultimate recipe to crash s2member. And if you have many users and sometimes need to copy/paste values because a transaction was lost - it’s not that unlikely to happen - the custom field where the information should have gone - is just below the cid field!

wp_s2member_subscr_cid:openmtbap.org

With openmtbmap.org usually being my custom value/domain. Another value does not cause problems. But my domain does. If I enter my domain to another user in the database - that user will then show up in the payment notifications - and that user will have data added to his profile!

Maybe this happened when I entered some data for a user whos subscription was lost due to the old paypal bug that sometimes users were not created. As soon as I add this into a line of my database the above mentioned bugs start happening!

Can you please try to reproduce it - if not I can send you a wp_users and wp_usermeta database table to play with. And then check why it causes s2member to run havoc and fix it. Because a wrongly entered value should NEVER let a program go crazy (stripe subscriptions not demoted, wrong payment api email sent, maybe more problems). And no that user is certainly not hardcoded…
Also the user changed over time - that is because I guess this happened several times - but once the user got demoted - the problem disappeared because the cid value will be deleted.

You can even enter the domain into a CID field of a subscriber - without any other info - it will cause the same problem!

Actually copying in your domain into the Paid Subscr. ID: field will cause the same mess.
And searching my database for 288 was one of my first things to do - it’s just that everything looked fine - on just reading over this field - I did not directly notice that it’s a wrong value. I was rather looking for empty field or some gibberish - but not just some correct information in the wrong field.

Nice work finding the culprit! :smiley:

Thank you for the details and sharing your findings.

Very interesting, too…

I dug in the code a bit, but haven’t spotted exactly where the mixup is happening yet. I’m suspecting it’s from one of the methods in s2member/src/includes/classes/utils-users.inc.php.

I’m making a note of this, to keep in mind to see how I can prevent it from happening to others in the future. :+1:

thanks - because likely I’m not the only one who did this mistake in the past. Some others reported they had problems with users not demoting (Stripe). For all the years we had the problem with paypal not waiting. That meant that about 1% of all users needed to be entered by hand. Then if you once copy in one field to high - the bug started. I have not yet confirmed that the users not being demoted is caused by this - but it seems likely that those fields at the wrong user - being constantly overwritten cause some mess. Even though all the visible fields were correct. I will check that up in a couple of days.

Bug:
(s2Member) Bug Fix: PayPal would sometimes return the customer without the Custom Value expected by s2Member, incorrectly triggering an error. A small delay has now been added when needed to wait for PayPal to provide the missing value, so that the customer is met with the correct success message on return. Props to Josh Hartman for his help.