Studying ultraluminous galaxies (ULIRGs) at early times (z > 2) provides insight into the formative phases of massive galaxies around us today. While the field has become a relatively mature science, there remain various aspects that are still mired in uncertainty, and new facilities offer possibilities for great leaps forward in our understanding. I will provide an overview of this exciting field, and present new avenues of research into high-z ULIRGs which are significantly pushing our understanding of the population. These include our recently completed IRAM-PdBI survey of CO in 50 high-z Sub-Mm Galaxies (SMGs), our VLT/Gemini IFU surveys of resolved nebular lines in SMGs, and the dust properties of these SMGs as measured with Herschel Space Telescope. I will also discuss the insights we are gaining from the lensed SMG population uncovered by the 2mm-selected South Pole Telescope (SPT) point sources.