No. Spelling, punctuation, and capitalization are all part of writing. Writing is not language -- it's the representation of language, which is spoken.

Is punctuation a language feature or structure?

The features of language that support meaning (for example, sentence structure, noun group/phrase, vocabulary, punctuation, figurative language, framing, camera angles).

Is punctuation part of the structure of a poem?

Punctuations within the verses of the poem are VERY important. Even where the poem has no punctuation, EXCLAMATION marks, at the end or within a verse, are needed to show the intensity of a verse.