If you can read Japanese and can get to Japan....then manga is extremely affordable. Used bookstores carry manga for anywhere from 100-700yen (700 being the big adult ones, and even that is pretty rare). Popular series, like Bleach, might be 300yen, but that's still under $3 per volume. Book-Off is a popular used bookstore branch that even has offices in the United States (I know there's one in LA and NYC, and I think Seattle but I'm not sure).
As for anime, I personally am one of those people who likes HAVING things. So in my case I find it worthwile. Of course, I don't actually own EVERYTHING that I also have in fansub form, but then, I'm still in college and don't have a steady income yet. Someday though, someday.