Perfume is activated by heat and chemical reactions. It's more effective when in contact with the body rather than the clothing. In fact, it can stain and damage some fabrics. Silk, for example, is an animal protein, which is prone to degradation due to the alcohol in perfume. Fabric dyes can bleed and fade. Alcohol evaporates a few minutes after application, however, and the fabric is preserved. On the other hand, synthetic fabrics such as polyester repel liquid ingredients, including alcohol, but are more susceptible to stains from perfume oils. Again, avoiding direct contact helps to minimize the problem.
Fragrances can be indirectly applied to fabrics, however, to safely scent your wardrobe. You can spritz perfume on padded clothes hangers, for instance. Or add a few drops of essential oils to the water when you rinse lingerie, or to the water in the iron to steam press handkerchiefs.
If you're handy at crafts, you can carry your favorite scent in a perfumed locket. Add a few drops of oil to melted beeswax, then pour the beeswax into an empty locket and let harden.