If you want to begin changing your actual financial information, you will just go to the Billing Information and Billing Address. Just go ahead and press down on the piece of information that you want to alter and you will be taken to a more detailed informational input section where you can change the information.
If you select your credit card number, you will be taken to a credit card screen where you can change not only the number but also the expiration date, security code, card type, card name, card address and personal phone number. Once you have changed the information you select Done as you would with any settings changes on your iPhone. If you select the card number or the billing address you will go to the same edit menu.
You may be concerned about making financial changes to your App Store account on your iPhone so you may want to check over again on iTunes from your desktop before you continue making purchases. There is also a section for Terms of Service and Privacy Policy so that you can see the standards for App Store security.