Currently one of the most established protocols for machine to machine (M2M) communications is LoRaWAN, designed to provide low power wide area network with features specifically needed to support low-cost, mobile, secure bidirectional communication for the Internet of Things (IoT). In this context security is of pivotal importance, as IoT constitutes a pervasive network of devices highly integrated with our daily life. In this paper we examine key security issues of the procedure used in LoRaWAN to allow an end device to establish a connection with the network server. We have identified vulnerabilities in this protocol, in particular with reference to the use of a random number in the join procedure packet, meant to prevent replay attacks. We first discuss the options that a network server has when detecting a replay attack and then we examine a) the possibility that a legitimate receiver is considered an attacker because of the random number generation issues and b) the possibility for an attacker to exploit this protocol to generate a denial of service (DoS). A wide set of experiments has been conducted using a widely used LoRaWAN chip showing the vulnerabilities of the protocol.