In Indo-European languages using a Latin alphabet, what’s up with these two letters “ch” that are pronounced (phonetics) so differently?

Roman alphabet digraphs were invented with the digraphs Latin used to represent Greek aspirated letters: <ch th ph>. So <ch> was available very very early on to languages using the Roman alphabet, to represent new sounds.

Palatal sounds are notoriously unstable phonologically: once /k/ goes to [c] (as it did in late Latin), it can then move on to any of [tɕ, tʃ, ʃ, s].

As a back consonant, <c> could be used to convey anything velar or palatal, or even palatoalveolar, given the possible targets of phonetic change for a fronted /k/.

• So <ch> could end up being conscripted as something velar—like a velar fricative /x/, in German.
• Or it can be used to mean that the velar is velar and not palatalised, like <chi> in Italian.
• Or it can be used to mean something palatal instead of velar—like the palatal stop /c/  in Old French.
• Or it can be used to represent any phoneme that the unstable /c/ ends up sounding like, including the palatoalveolar /ʃ/ in Modern French, or /tʃ/ in English and Spanish.