I've managed to resurrect my old iPhone 3G which runs 4.2.1. Can I still develop using XCode on this phone? I would ideally like to use it for learning Obj-C without paying for the Developers Program, Jailbreaking just to test my apps.
Is this feasible?
Yes it is feasible but you will not be able to use xcode the latest xcode 4.5 with it because it supports ios 4.3 and later. You would be able to use lower versions of xcode for 4.2.1 . The latest release notes indicates 4.2.1 and lower will not be supported, we now have to use 2 version of Xcode to develop when supporting older devices.
Please go through
Old Xcode
Xcode 4.5 and iOS 4.2.1 incompatibility
XCode 4 and iOS 4.2.1