Definitely works here.
1. I clicked SORT MODE so it was set to CUSTOM.
2. I have dragged images around randomly in custom order:

3. Clicked SAVE.
4. View the specific page, and you can see the order is the same as I have sorted in:
[LINK REMOVED]
In this page gallery sort settings, I notice you had set DESCENDING. That means it would order images REVERSE of your custom order. This is not really logical form CUSTOM ordering, so I changed it to ASCENDING:
I have tried with Firefox, Edge and Chrome all on Win10 Pro and Win7 Enterprise.
Really not sure what the problem is, but it seems to be on your side. Just to avoid any confusion, you are aware that you must be in
CUSTOM sort mode in the panel (se my screenshot) to be able to DRAG images? Just because you have set "custom" sort, doesn't mean the panel doesn't allow viewing images in name/date/size orders.
I have also noticed that when I log back in the control panel, the gallery is displayed by name and not custom, like if the Sort by Custom setting doesn't stay in memory.
Not quite sure what you mean. When I refresh the page, it still displays the saved custom sort order:
If I change the view to another ORDER, then yes the order will change in panel of course, but you have still selected custom sort order for this gallery.