{"cells":[{"cell_type":"markdown","id":"5db15ce8","metadata":{"id":"5db15ce8"},"source":["---\n","<font size = 6><center><font face = garamond> <b> Mathematics in the context of AI</center>\n","<font size = 5><center><font face = garamond> <b> Markov Chain Sentence Generator</center>\n","\n","---\n","\n","Joaquin Carbonara 4/28/2024\n","\n","---\n","\n"]},{"cell_type":"markdown","id":"2d1f16b9","metadata":{"id":"2d1f16b9"},"source":["## Introduction\n","This notebook demonstrates how to read a book from Project Gutenberg and use it to train a Markov chain model for generating random sentences. A Markov chain is a stochastic model describing a sequence of possible events in which the probability of each event depends only on the state attained in the previous event. In the context of language, it can be used to generate sentences that resemble the style of the source text."]},{"cell_type":"code","execution_count":1,"id":"930cc1d2","metadata":{"id":"930cc1d2","executionInfo":{"status":"ok","timestamp":1714308540651,"user_tz":240,"elapsed":2,"user":{"displayName":"Joaquin Carbonara","userId":"14921749483686740647"}}},"outputs":[],"source":["# Importing necessary libraries\n","import requests\n","import random"]},{"cell_type":"code","execution_count":2,"id":"bab2e7d6","metadata":{"id":"bab2e7d6","executionInfo":{"status":"ok","timestamp":1714308566412,"user_tz":240,"elapsed":2,"user":{"displayName":"Joaquin Carbonara","userId":"14921749483686740647"}}},"outputs":[],"source":["# Function to download a book from Project Gutenberg\n","def download_book(url):\n","    try:\n","        response = requests.get(url)\n","        response.raise_for_status()\n","        return response.text\n","    except requests.RequestException as e:\n","        print(f'Error downloading the book: {e}')\n","        return None\n","\n","# Function to build a Markov chain model from text\n","def build_markov_chain(text, chain={}):\n","    words = text.split()\n","    index = 1\n","\n","    for word in words[index:]:\n","        key = words[index - 1]\n","        if key in chain:\n","            chain[key].append(word)\n","        else:\n","            chain[key] = [word]\n","        index += 1\n","\n","    return chain\n","\n","# Function to generate a random sentence\n","def generate_sentence(chain, count=15):\n","    word1 = random.choice(list(chain.keys()))\n","    sentence = word1.capitalize()\n","\n","    for i in range(count-1):\n","        word2 = random.choice(chain[word1])\n","        word1 = word2\n","        sentence += ' ' + word2\n","\n","    sentence += '.'\n","    return sentence"]},{"cell_type":"code","execution_count":3,"id":"c1c9fc1f","metadata":{"id":"c1c9fc1f","colab":{"base_uri":"https://localhost:8080/"},"executionInfo":{"status":"ok","timestamp":1714308572916,"user_tz":240,"elapsed":2509,"user":{"displayName":"Joaquin Carbonara","userId":"14921749483686740647"}},"outputId":"6ada6924-0551-47e8-bfc7-0b7daf48b1cc"},"outputs":[{"output_type":"stream","name":"stdout","text":["Generated Sentence: Eliza? Do not afraid you can’t write; so accomplished! My poor sister. Before any message.\n"]}],"source":["# Main function to execute the script\n","def main():\n","    # URL of a book from Project Gutenberg\n","    book_url = 'https://www.gutenberg.org/files/1342/1342-0.txt'  # Example: Pride and Prejudice by Jane Austen\n","\n","    # Download the book\n","    book_text = download_book(book_url)\n","\n","    # Check if the book was downloaded successfully\n","    if book_text:\n","        # Build the Markov chain model\n","        markov_chain = build_markov_chain(book_text)\n","\n","        # Generate and print a random sentence\n","        random_sentence = generate_sentence(markov_chain)\n","        print('Generated Sentence:', random_sentence)\n","    else:\n","        print('Failed to download the book.')\n","\n","# Run the main function\n","if __name__ == '__main__':\n","    main()"]},{"cell_type":"markdown","id":"dae45a49","metadata":{"id":"dae45a49"},"source":["## Understanding Markov Chains in Sentence Generation\n","In this notebook, the Markov chain model is used to generate sentences. Each word in the generated sentence is based on the probability of that word following the preceding word, according to the data from the book used to train the model. This is a fundamental property of Markov chains where the next state (or word, in this case) depends only on the current state and not on the sequence of events that preceded it. The `build_markov_chain` function constructs this model by mapping each word to the list of words that follow it in the source text, thus creating a simple but effective Markov chain for text generation."]},{"cell_type":"markdown","source":["\n","----\n","\n","<font size = 5><center><font face = garamond> <b> Entropy of the value of a key word in the Markov Chain dictionary\n","</center>\n","\n","----"],"metadata":{"id":"pArA5s3eOCIf"},"id":"pArA5s3eOCIf"},{"cell_type":"code","source":["import requests\n","from collections import Counter\n","import math\n","\n","word='sly'\n","\n","book_url = 'https://www.gutenberg.org/files/1342/1342-0.txt'  # Example: Pride and Prejudice by Jane Austen\n","\n","# Download the book\n","book_text = download_book(book_url)\n","\n","# Check if the book was downloaded successfully\n","if book_text:\n","    # Build the Markov chain model\n","    markov_chain = build_markov_chain(book_text)"],"metadata":{"id":"NjZnByMcM2Gh","executionInfo":{"status":"ok","timestamp":1714266447966,"user_tz":240,"elapsed":896,"user":{"displayName":"Joaquin Carbonara","userId":"14921749483686740647"}}},"id":"NjZnByMcM2Gh","execution_count":21,"outputs":[]},{"cell_type":"code","source":["from collections import Counter\n","words=markov_chain[word]\n","\n","# Count the frequency of each word\n","word_counts = Counter(words)\n","\n","# Calculate the probability of each word\n","total_words = len(words)\n","word_probabilities = {word: count / total_words for word, count in word_counts.items()}\n","\n","# Calculate the entropy\n","entropy = 0\n","for probability in word_probabilities.values():\n","    if probability > 0:\n","        entropy += probability * math.log(1 / probability, 2)\n","max_ent=math.log(total_words,2)\n","print (\"| entropy of\", word,\":\",entropy,\"| max_entropy :\",max_ent,\"| rel_entropy :\",entropy/max_ent,\"|\")"],"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"id":"6mBbRwqqNXNJ","executionInfo":{"status":"ok","timestamp":1714267664523,"user_tz":240,"elapsed":155,"user":{"displayName":"Joaquin Carbonara","userId":"14921749483686740647"}},"outputId":"235cc0e0-fb23-4fde-eccc-7e5e95b41847"},"id":"6mBbRwqqNXNJ","execution_count":30,"outputs":[{"output_type":"stream","name":"stdout","text":["| entropy of sly : 1.0 | max_entropy : 3.5849625007211565 | rel_entropy : 0.2789429456511298 |\n"]}]}],"metadata":{"colab":{"provenance":[]},"language_info":{"name":"python"},"kernelspec":{"name":"python3","display_name":"Python 3"}},"nbformat":4,"nbformat_minor":5}