Based on this excerpt, what can be inferred about oliver’s neighbors?

What is implied in this sentence from mark twain's "the ? 1,000,000 bank-note"? i was puzzled, and wanted to discuss the matter a little further, but they didn't; so i took my leave, feeling hurt and insulted to be made the butt of what was apparently some kind of a practical joke, and yet obliged to put up with it, not being in circumstances to resent affronts from rich and strong folk.