Welcome to Japan!
Most onsen are attached to hotels/ryokans, so you need to be a guest to use them. There are “higaeri onsen” which are “day-trip onsen” but there aren’t as many.
Many onsen hotels have a “kashikiri-buro” or a reserved/rental bath which is basically a private bathing area. While not all of them are accessible, some are. However, you need to be staying at the hotel to be able to reserve it.
Since you mentioned you have arrived already, I assume that you have booked all of your hotels? If so, you may not be able to go without cancelling your current reservation and booking at the onsen.
Would a sento (public bath) be OK? These are baths where you can pay to use them and they aren’t connected with hotels. There is an accessible one near Ueno with a “family bath” where you and your wife can bathe together. It is 1500 yen per person for 90minutes. You need to call to reserve it. Here is the website:
http://mikokuyu.com/
(Be careful as they are closed on Mondays in August.)
You can see some pictures on this site (Japanese only):
https://www.media116.jp/outing/2894
If sento (not onsen) like this are OK, I can look for others in other cities.