To people of loved ones with poor memory:

It might be frustrating when they forget something (especially if it's important), but I can guarantee you that they are already beating themself up over having forgotten yet another thing. You don't need to rub that shame, embarrassment, and humiliation in deeper than it already is.

It is scary to forget things. It's humiliating to be told that the only reason we forget things is because we think nothing is important, that we're selfish and callous. Our brains are being pulled in every direction at the same time. It's impossible to keep track of this shit every picosecond of the day.

People like to conceptualize memory issues as a matter of lesser intelligence, that we're too stupid to even remember [minor detail]. I've noticed, though, that all of our brain power is kept toward other things - keeping ourselves alive, remembering a different thing, trying to regulate emotions or other disorders. Nobody seems to care that our workload is at least twice that of the "average" person's, I guess because they often don't directly notice it, or it doesn't directly affect them?

It's fine to be upset about the situation. You can't help that reaction, but you do not have to be cruel to people with memory issues, no matter the cause of it. Whatever they forgot might have been important to you, but there may be other things in that person's life that required their brainpower.