Is it an ad-hominem when I get called "a pessimist who won't be happy with positive changes in situation X, so further debate is pointless", even though I've presented my arguments for why I'm skeptical of any positive changes in situation X? I feel like it's a dismissive tactic, but would like some clarification.

This is a good, and difficult question. There's no doubt that, in some cases, this sort of objection really is an unfair, unhelpful dismissal. Calling you names ("silly pessimist!") could just be a way to fail to engage with what you're saying. Though that may be what's going on in your case, sometimes such objections actually do have a point. Let's set aside the "ad-hominem" fallacy, whether this is an instance of it, and what that implies, and just consider whether one might have a good point when one objects with something like "you just think that because you're a pessimist." It is useful to compare this with a more straightforward case first. Suppose you think that your daughter is the best singer in the choir, and someone says to you "you just think that because she's your daughter!" You might reply, "but I hear her singing, and I hear the others, and I genuinely think she's got the best voice." "Yeah," the objector replies, "of course you think that, whatever, have fun thinking she's the best!"...