<p>I'm not sure if this qualifies as a bug, so I'll let these examples speak for themselves. Note that I'm (ab)using the
default tag to render these infobox fragments inline.
</p>
<h2><span class="mw-headline" id="Example_1"> Example 1 </span></h2>
<aside class="portable-infobox pi-background pi-theme-wikia pi-layout-default"><figure class="pi-item pi-image">

<figcaption class="pi-item-spacing pi-caption">This caption is defined inside the caption tag.
Wikitext works.</figcaption></figure></aside>
<infobox>
<image>
<default>[[File:Example.jpg]]</default>
<caption>
<default>This caption is defined inside the <code>caption</code> tag. [[Help:Wikitext|Wikitext]] '''works'''.</default>
</caption>
</image>
</infobox>
<h2><span class="mw-headline" id="Example_2"> Example 2 </span></h2>
<aside class="portable-infobox pi-background pi-theme-wikia pi-layout-default"><figure class="pi-item pi-image">

<figcaption class="pi-item-spacing pi-caption">This caption is defined inside the caption tag. [[Help:Wikitext|Wikitext]] '''works'''.</figcaption></figure></aside>
<infobox>
<image>
<default>
<gallery>
Example.jpg|This caption is defined inside the <code>caption</code> tag. [[Help:Wikitext|Wikitext]] '''works'''.
</gallery>
</default>
<caption>
<default>This caption is defined inside the <code>gallery</code> tag. It is invisible.</default>
</caption>
</image>
</infobox>
<h2><span class="mw-headline" id="Example_3"> Example 3 </span></h2>
<aside class="portable-infobox pi-background pi-theme-wikia pi-layout-default"><div class="pi-image-collection">
-
Tab, '''''not''''' caption
-
Tab, '''''not''''' caption
<div class="pi-image-collection-tab-content current" id="pi-tab-0">
<figure class="pi-item pi-image">

</figure></div>
<div class="pi-image-collection-tab-content " id="pi-tab-1">
<figure class="pi-item pi-image">

</figure></div>
</div>
</aside>
<infobox>
<image>
<default>
<gallery>
Example.jpg|Tab, '''''not''''' caption
Example.jpg|Tab, '''''not''''' caption
</gallery>
</default>
<caption>
<default>This caption is defined inside the <code>caption</code> tag. It is invisible.</default>
</caption>
</image>
</infobox>