If you are experiencing issues with your WordPress caption text not showing properly and the images themselves not aligning correctly, then you know how annoying this problem can be.  A good caption always complements a picture or graphic and helps draw readers into the actual article or post.  Not being able to utilize such a useful tool effectively is quite frustrating.  

The root cause of this issue has to do with a deliberate change made in WordPress version 3.4. Instead of caption="" being an attribute of the [caption] shortcode, WordPress 3.4+ inserts the caption in the main content of the shortcode, right after the image and between [caption] and [/caption]. This change was made to allow for HTML in captions. In older versions of certain themes, such as Thesis and Headway, this change has resulted in improperly formatted captions and aligned images because of the themes are not using the appropriate 'img_caption_shortcode' filter.

The long term solution to the issue is to upgrade your theme to the latest version; however, since theme upgrades can be a time-intensive process, we have detailed a quick and easy temporary fix below.

Step #1
Locate the caption code in the appropriate post or page and change:

[caption id="attachment_123" align="aligncenter" width="550"]<img src="...">My Caption[/caption]

to

[caption id="attachment_123" align="aligncenter" width="550" caption="My Caption"]<img src="...">[/caption]

by editing to include the caption text within the [caption] shortcode instead of after the html for the image.

Step #2
Now refresh the page. Your image and caption should now be displayed correctly.

More from the WordPress Development Team (via the community forum):

This change in 3.4 was necessary to allow for HTML in captions. But, like everything we develop, it was deliberately coded so it would be backwards compatible. Any theme using the 'img_caption_shortcode' filter to override how a caption is generated would work swimmingly, even though the format changed.

If I had to guess, the common issue for all for these themes is that they call add_shortcode( 'caption', 'some_function_in_their_theme' ); to customize the caption, instead of using the aforementioned filter. This is wrong. Themes should not be doing this. If the theme you are using does this, please contact the theme author. If they disagree it is a problem with their theme, I’d be happy to explain it to them.