It's an oddball question, but but bear with me. Driving north this morning and watched an AS 737 in new livery take off towards west from PDX. From my distance of about a mile, and the angle of light, it looked easily mistakable the new LH scheme. Got me to wondering if there is regulatory oversight to maintain differentiation?
With regard to your question, I don’t think there’d be any regulation preventing an airline from painting in a similar scheme to another airline anymore than there’d be a regulation which stops a local burger restaurant owner from painting his store in similar colors to McDonald’s. However, the airline being copied would likely have the right to sue as McDonald’s would in the burger joint analogy.
Speaking to the broader point of liveries, yes there are regulations on branding. A case I’m extremely familiar with is that of Caribbean Airlines/Air Jamaica explained here https://worldairlinenews.com/2012/07/22 ... brand/amp/
That was enforced by the Trinidad and Tobago Civil Aviation Authority. However, I do believe similar policies are in place in most countries. Neither American nor Delta for example could simply decide tomorrow to change their brand name to “Amazing Airlines” and start painting their planes accordingly without seeking approval for the name change.