Construction of Oxford castle began just a couple of years after the Norman conquest, by the Norman baron Robert D'Oyly. The castle uses a stream of the Thames as part of it's defences. The stream was diverted to provide the castle's moat. The original castle was a motte and bailey type, originally made of wood but with stone replacing the original walls and buildings in the 12th and 13th centuries. The surviving St George's tower may pre-date the rest of the castle and be of Saxon origin.
The castle saw action in the 12 and 13th centuries including during the Baron's War. The castle lost it's military role in the late 14th century and began to decay. However, it was bought back into action during the Civil War. This was the last time the castle was used in war, though it did have a later role as Oxford prison! It continued in this role until 1996 though much of the castle structure was lost over the years including the keep in the 18th century.