The current Imperial Palace is located on the grounds of the former Edo Castle (江戸城, Edo-jo), and is still surrounded by the same stone walls and moat today. The Imperial Palace was destroyed by air raids during the second World War, but was reconstructed in the same design after. Only the East Gardens are open to the general public, with the inner areas being off-limit. There are guided tours that take you into certain parts of the inner area, and you can find out more info on that here.
Edo Castle was the residence of the shogun (将軍) during the Edo Period (1603-1867, 江戸時代, Edo-jidai), followed by the Emperor (天皇, Tennō, Heavenly Sovereign) from 1867 till 1888 when the current Imperial Palace was built.
During that time, Edo Castle only had a tenshu (天守, Main Keep) for 19 years from 1638 to 1657. When it was built, it was the tallest tenshu in Japan, but it was destroyed in a fire in 1657 and has not been rebuilt since. Today, the stone foundations of the castle keep still remain, and serve as a observation point for the garden.
The area of the former Edo Castle have been turned into what’s known as the Imperial Palace East Gardens today, and they occupy what used to be known as the hon-maru (本丸, Innermost Complex) and ni-no-maru (二の丸, Second Inner Complex) of the old castle. None of the original castle buildings still exist today, but the moat and gates are still the originals from the Edo Castle.
