The reason is because of punched cards, a method of data storage now no longer used. With punched cards binary data was encoded by a hole in the card or the lack of one. The best way to tell a computer to ignore a character (or delete it) was to punch all of the holes in that character which in those days was encoded in seven bits. Therefore 1111111 (in binary) was delete. In hexadecimal that is 7F which is where the DEL control code is in the chart below.
|Public domain image from Wikipedia|
* Many of these control codes were intended for teletypes and have little utility these days.