With few notable exceptions, over the centuries since the destruction of the second Holy Temple, very few rabbis have studied and commented on the Mishnaic and Talmudic source material concerning the Holy Temple.
Among the few that did dedicate themselves to studying about the Holy Temple, was the conviction that rebuilding the Holy Temple was a Torah imperative, that must be and can be achieved by man.
The general lack of study, however, may explain many of the misconceptions held today.