Given the United States' continuing struggles in international rugby at all levels, Armchair Playmaker though it might be helpful to provide a look at some baseline comparisons. The International Rugby Board's site has great player registration statistics, etc., easily available for participating national unions, and we've used them to create this table comparing how many people play rugby union across the nations.
For parsimony, we've included only figures for each country who has been a Rugby World Cup champion or runner-up since the tournament's inception in 1987, along with the United States. We've also used these figures to calculate the percentage of each nation's population that plays rugby union, the number of referees, and the number of referees per player. Click the chart below for a closer look:

As you can see, all of the nations listed but Australia have at least twice as many players in the nation as the United States does. England has more than ten times as many. Additionally, the gap in popularity by proportion is perhaphs more instructive. Every World Cup winner but Australia have at least one registered rugby union player for every 100 people in their population. In rugby-mad New Zealand, three of every 100 souls play. France, which has been in two finals but hasn't yet won the trophy, is a bit lower with three in a thousand.
Meanwhile, the United States has one rugby union player for every five thousand people in their population. Of course, these figures don't take into account factors such as what level these players are playing at, how many are male and female, what the development infrastructure is, etc., but the general picture should give a little bit of an idea of the challenge the U.S. side faces in terms of player pools.
Interestingly, the U.S. number of referees per player rate is not too far off compared to a lot of nations, but you may want to consider that this won't reflect the rate of full-time referees, etc., which is very low in the United States.
Some have called the United States a "sleeping giant" in rugby union (though one of those people has said a few less promising things since). If that's the case, these figures give an idea of how deep the slumber is.
Just for fun, we've also compared the figures for the World Cup winners and runners-up to how many players there are per World Cup win to see who's getting the best return on their player resources. Click to have a look:

Of course, these numbers don't take into count a lot of factors. The present numbers don't say anything about how many people played in a nation several years ago, for example. The nature of the game and player development have also changed a lot since the advent of professionalism in 1995, which was after three World Cups had already been played. Considering these and other factors, that last chart needs to be taken with a grain of salt. All the same, it looks like whatever Australia is doing, they need to keep doing it! England has over 20 times as many players per World Cup win, and even New Zealand has four times as many.