+*function* : The binary function to apply. It takes two arguments and returns a single value.
+*iterable* : The sequence of elements to process.
+*initial (optional)*: An initial value. If provided, the function is applied to the initial value and the first element of the iterable. Otherwise, the first two elements are used as the initial values. + +## Working: +- Intially , first two elements of iterable are picked and the result is obtained. +- Next step is to apply the same function to the previously attained result and the number just succeeding the second element and the result is again stored. +- This process continues till no more elements are left in the container. +- The final returned result is returned and printed on console. + +## Examples: + +**Example 1:** +```python +numbers = [1, 2, 3, 4, 10] +total = reduce(lambda x, y: x + y, numbers) +print(total) # Output: 20 +``` +**Example 2:** +```python +numbers = [11, 7, 8, 20, 1] +max_value = reduce(lambda x, y: x if x > y else y, numbers) +print(max_value) # Output: 20 +``` +**Example 3:** +```python +# Importing reduce function from functools +from functools import reduce + +# Creating a list +my_list = [10, 20, 30, 40, 50] + +# Calculating the product of the numbers in my_list +# using reduce and lambda functions together +product = reduce(lambda x, y: x * y, my_list) + +# Printing output +print(f"Product = {product}") # Output : Product = 12000000 +``` + +## Difference Between reduce() and accumulate(): +- **Behavior:** + - reduce() stores intermediate results and only returns the final summation value. + - accumulate() returns an iterator containing all intermediate results. The last value in the iterator is the summation value of the list. + +- **Use Cases:** + - Use reduce() when you need a single result (e.g., total sum, product) from the iterable. + - Use accumulate() when you want to access intermediate results during the reduction process. + +- **Initial Value:** + - reduce() allows an optional initial value. + - accumulate() also accepts an optional initial value since Python 3.8. + +- **Order of Arguments:** + - reduce() takes the function first, followed by the iterable. + - accumulate() takes the iterable first, followed by the function. + +## Conclusion: +Python's Reduce function enables us to apply reduction operations to iterables using lambda and callable functions. A +function called reduce() reduces the elements of an iterable to a single cumulative value. The reduce function in +Python solves various straightforward issues, including adding and multiplying iterables of numbers. \ No newline at end of file diff --git a/contrib/advanced-python/regular_expressions.md b/contrib/advanced-python/regular_expressions.md index 65ff2c2b..81c883ec 100644 --- a/contrib/advanced-python/regular_expressions.md +++ b/contrib/advanced-python/regular_expressions.md @@ -1,36 +1,144 @@ ## Regular Expressions in Python -Regular expressions (regex) are a powerful tool for pattern matching and text manipulation. +Regular expressions (regex) are a powerful tool for pattern matching and text manipulation. Python's re module provides comprehensive support for regular expressions, enabling efficient text processing and validation. +Regular expressions (regex) are a versitile tool for matching patterns in strings. In Python, the `re` module provides support for working with regular expressions. ## 1. Introduction to Regular Expressions -A regular expression is a sequence of characters defining a search pattern. Common use cases include validating input, searching within text, and extracting +A regular expression is a sequence of characters defining a search pattern. Common use cases include validating input, searching within text, and extracting specific patterns. ## 2. Basic Syntax Literal Characters: Match exact characters (e.g., abc matches "abc"). -Metacharacters: Special characters like ., *, ?, +, ^, $, [ ], and | used to build patterns. +Metacharacters: Special characters like ., \*, ?, +, ^, $, [ ], and | used to build patterns. **Common Metacharacters:** -* .: Any character except newline. -* ^: Start of the string. -* $: End of the string. -* *: 0 or more repetitions. -* +: 1 or more repetitions. -* ?: 0 or 1 repetition. -* []: Any one character inside brackets (e.g., [a-z]). -* |: Either the pattern before or after. - +- .: Any character except newline. +- ^: Start of the string. +- $: End of the string. +- *: 0 or more repetitions. +- +: 1 or more repetitions. +- ?: 0 or 1 repetition. +- []: Any one character inside brackets (e.g., [a-z]). +- |: Either the pattern before or after. +- \ : Used to drop the special meaning of character following it +- {} : Indicate the number of occurrences of a preceding regex to match. +- () : Enclose a group of Regex + +Examples: + +1. `.` + +```bash +import re +pattern = r'c.t' +text = 'cat cot cut cit' +matches = re.findall(pattern, text) +print(matches) # Output: ['cat', 'cot', 'cut', 'cit'] +``` + +2. `^` + +```bash +pattern = r'^Hello' +text = 'Hello, world!' +match = re.search(pattern, text) +print(match.group() if match else 'No match') # Output: 'Hello' +``` + +3. `$` + +```bash +pattern = r'world!$' +text = 'Hello, world!' +match = re.search(pattern, text) +print(match.group() if match else 'No match') # Output: 'world!' +``` + +4. `*` + +```bash +pattern = r'ab*' +text = 'a ab abb abbb' +matches = re.findall(pattern, text) +print(matches) # Output: ['a', 'ab', 'abb', 'abbb'] +``` + +5. `+` + +```bash +pattern = r'ab+' +text = 'a ab abb abbb' +matches = re.findall(pattern, text) +print(matches) # Output: ['ab', 'abb', 'abbb'] +``` + +6. `?` + +```bash +pattern = r'ab?' +text = 'a ab abb abbb' +matches = re.findall(pattern, text) +print(matches) # Output: ['a', 'ab', 'ab', 'ab'] +``` + +7. `[]` + +```bash +pattern = r'[aeiou]' +text = 'hello world' +matches = re.findall(pattern, text) +print(matches) # Output: ['e', 'o', 'o'] +``` + +8. `|` + +```bash +pattern = r'cat|dog' +text = 'I have a cat and a dog.' +matches = re.findall(pattern, text) +print(matches) # Output: ['cat', 'dog'] +``` + +9. `\`` + +```bash +pattern = r'\$100' +text = 'The price is $100.' +match = re.search(pattern, text) +print(match.group() if match else 'No match') # Output: '$100' +``` + +10. `{}` + +```bash +pattern = r'\d{3}' +text = 'My number is 123456' +matches = re.findall(pattern, text) +print(matches) # Output: ['123', '456'] +``` + +11. `()` + +```bash +pattern = r'(cat|dog)' +text = 'I have a cat and a dog.' +matches = re.findall(pattern, text) +print(matches) # Output: ['cat', 'dog'] +``` + ## 3. Using the re Module **Key functions in the re module:** -* re.match(): Checks for a match at the beginning of the string. -* re.search(): Searches for a match anywhere in the string. -* re.findall(): Returns a list of all matches. -* re.sub(): Replaces matches with a specified string. +- re.match(): Checks for a match at the beginning of the string. +- re.search(): Searches for a match anywhere in the string. +- re.findall(): Returns a list of all matches. +- re.sub(): Replaces matches with a specified string. +- re.split(): Returns a list where the string has been split at each match. +- re.escape(): Escapes special character + Examples: -Examples: ```bash import re @@ -45,12 +153,20 @@ print(re.findall(r'\d+', 'abc123def456')) # Output: ['123', '456'] # Substitute matches print(re.sub(r'\d+', '#', 'abc123def456')) # Output: abc#def# + +#Return a list where it get matched +print(re.split("\s", txt)) #['The', 'Donkey', 'in', 'the','Town'] + +# Escape special character +print(re.escape("We are good to go")) #We\ are\ good\ to\ go ``` ## 4. Compiling Regular Expressions + Compiling regular expressions improves performance for repeated use. Example: + ```bash import re @@ -58,12 +174,15 @@ pattern = re.compile(r'\d+') print(pattern.match('123abc').group()) # Output: 123 print(pattern.search('abc123').group()) # Output: 123 print(pattern.findall('abc123def456')) # Output: ['123', '456'] + ``` ## 5. Groups and Capturing + Parentheses () group and capture parts of the match. Example: + ```bash import re @@ -76,21 +195,46 @@ if match: ``` ## 6. Special Sequences + Special sequences are shortcuts for common patterns: -* \d: Any digit. -* \D: Any non-digit. -* \w: Any alphanumeric character. -* \W: Any non-alphanumeric character. -* \s: Any whitespace character. -* \S: Any non-whitespace character. +- \A:Returns a match if the specified characters are at the beginning of the string. +- \b:Returns a match where the specified characters are at the beginning or at the end of a word. +- \B:Returns a match where the specified characters are present, but NOT at the beginning (or at the end) of a word. +- \d: Any digit. +- \D: Any non-digit. +- \w: Any alphanumeric character. +- \W: Any non-alphanumeric character. +- \s: Any whitespace character. +- \S: Any non-whitespace character. +- \Z:Returns a match if the specified characters are at the end of the string. + Example: + ```bash import re print(re.search(r'\w+@\w+\.\w+', 'Contact: support@example.com').group()) # Output: support@example.com ``` +## 7.Sets + +A set is a set of characters inside a pair of square brackets [] with a special meaning: + +- [arn] : Returns a match where one of the specified characters (a, r, or n) is present. +- [a-n] : Returns a match for any lower case character, alphabetically between a and n. +- [^arn] : Returns a match for any character EXCEPT a, r, and n. +- [0123] : Returns a match where any of the specified digits (0, 1, 2, or 3) are present. +- [0-9] : Returns a match for any digit between 0 and 9. +- [0-5][0-9] : Returns a match for any two-digit numbers from 00 and 59. +- [a-zA-Z] : Returns a match for any character alphabetically between a and z, lower case OR upper case. +- [+] : In sets, +, \*, ., |, (), $,{} has no special meaning +- [+] means: return a match for any + character in the string. + ## Summary -Regular expressions are a versatile tool for text processing in Python. The re module offers powerful functions and metacharacters for pattern matching, -searching, and manipulation, making it an essential skill for handling complex text processing tasks. + +Regular expressions (regex) are a powerful tool for text processing in Python, offering a flexible way to match, search, and manipulate text patterns. The re module provides a comprehensive set of functions and metacharacters to tackle complex text processing tasks. +With regex, you can: +1.Match patterns: Use metacharacters like ., \*, ?, and {} to match specific patterns in text. +2.Search text: Employ functions like re.search() and re.match() to find occurrences of patterns in text. +3.Manipulate text: Utilize functions like re.sub() to replace patterns with new text. diff --git a/contrib/advanced-python/threading.md b/contrib/advanced-python/threading.md new file mode 100644 index 00000000..fa315335 --- /dev/null +++ b/contrib/advanced-python/threading.md @@ -0,0 +1,198 @@ +# Threading in Python +Threading is a sequence of instructions in a program that can be executed independently of the remaining process and +Threads are like lightweight processes that share the same memory space but can execute independently. +The process is an executable instance of a computer program. +This guide provides an overview of the threading module and its key functionalities. + +## Key Characteristics of Threads: +* Shared Memory: All threads within a process share the same memory space, which allows for efficient communication between threads. +* Independent Execution: Each thread can run independently and concurrently. +* Context Switching: The operating system can switch between threads, enabling concurrent execution. + +## Threading Module +This module will allows you to create and manage threads easily. This module includes several functions and classes to work with threads. + +**1. Creating Thread:** +To create a thread in Python, you can use the Thread class from the threading module. + +Example: +```python +import threading + +# Create a thread +thread = threading.Thread() + +# Start the thread +thread.start() + +# Wait for the thread to complete +thread.join() + +print("Thread has finished execution.") +``` +Output : +``` +Thread has finished execution. +``` +**2. Performing Task with Thread:** +We can also perform a specific task by thread by giving a function as target and its argument as arg ,as a parameter to Thread object. + +Example: + +```python +import threading + +# Define a function that will be executed by the thread +def print_numbers(arg): + for i in range(arg): + print(f"Thread: {i}") +# Create a thread +thread = threading.Thread(target=print_numbers,args=(5,)) + +# Start the thread +thread.start() + +# Wait for the thread to complete +thread.join() + +print("Thread has finished execution.") +``` +Output : +``` +Thread: 0 +Thread: 1 +Thread: 2 +Thread: 3 +Thread: 4 +Thread has finished execution. +``` +**3. Delaying a Task with Thread's Timer Function:** +We can set a time for which we want a thread to start. Timer function takes 4 arguments (interval,function,args,kwargs). + +Example: +```python +import threading + +# Define a function that will be executed by the thread +def print_numbers(arg): + for i in range(arg): + print(f"Thread: {i}") +# Create a thread after 3 seconds +thread = threading.Timer(3,print_numbers,args=(5,)) + +# Start the thread +thread.start() + +# Wait for the thread to complete +thread.join() + +print("Thread has finished execution.") +``` +Output : +``` +# after three second output will be generated +Thread: 0 +Thread: 1 +Thread: 2 +Thread: 3 +Thread: 4 +Thread has finished execution. +``` +**4. Creating Multiple Threads** +We can create and manage multiple threads to achieve concurrent execution. + +Example: +```python +import threading + +def print_numbers(thread_name): + for i in range(5): + print(f"{thread_name}: {i}") + +# Create multiple threads +thread1 = threading.Thread(target=print_numbers, args=("Thread 1",)) +thread2 = threading.Thread(target=print_numbers, args=("Thread 2",)) + +# Start the threads +thread1.start() +thread2.start() + +# Wait for both threads to complete +thread1.join() +thread2.join() + +print("Both threads have finished execution.") +``` +Output : +``` +Thread 1: 0 +Thread 1: 1 +Thread 2: 0 +Thread 1: 2 +Thread 1: 3 +Thread 2: 1 +Thread 2: 2 +Thread 2: 3 +Thread 2: 4 +Thread 1: 4 +Both threads have finished execution. +``` + +**5. Thread Synchronization** +When we create multiple threads and they access shared resources, there is a risk of race conditions and data corruption. To prevent this, you can use synchronization primitives such as locks. +A lock is a synchronization primitive that ensures that only one thread can access a shared resource at a time. + +Example: +```Python +import threading + +lock = threading.Lock() + +def print_numbers(thread_name): + for i in range(10): + with lock: + print(f"{thread_name}: {i}") + +# Create multiple threads +thread1 = threading.Thread(target=print_numbers, args=("Thread 1",)) +thread2 = threading.Thread(target=print_numbers, args=("Thread 2",)) + +# Start the threads +thread1.start() +thread2.start() + +# Wait for both threads to complete +thread1.join() +thread2.join() + +print("Both threads have finished execution.") +``` +Output : +``` +Thread 1: 0 +Thread 1: 1 +Thread 1: 2 +Thread 1: 3 +Thread 1: 4 +Thread 1: 5 +Thread 1: 6 +Thread 1: 7 +Thread 1: 8 +Thread 1: 9 +Thread 2: 0 +Thread 2: 1 +Thread 2: 2 +Thread 2: 3 +Thread 2: 4 +Thread 2: 5 +Thread 2: 6 +Thread 2: 7 +Thread 2: 8 +Thread 2: 9 +Both threads have finished execution. +``` + +A ```lock``` object is created using threading.Lock() and The ```with lock``` statement ensures that the lock is acquired before printing and released after printing. This prevents other threads from accessing the print statement simultaneously. + +## Conclusion +Threading in Python is a powerful tool for achieving concurrency and improving the performance of I/O-bound tasks. By understanding and implementing threads using the threading module, you can enhance the efficiency of your programs. To prevent race situations and maintain data integrity, keep in mind that thread synchronization must be properly managed. diff --git a/contrib/advanced-python/type-hinting.md b/contrib/advanced-python/type-hinting.md new file mode 100644 index 00000000..fcf1e1c0 --- /dev/null +++ b/contrib/advanced-python/type-hinting.md @@ -0,0 +1,106 @@ +# Introduction to Type Hinting in Python +Type hinting is a feature in Python that allows you to specify the expected data types of variables, function arguments, and return values. It was introduced +in Python 3.5 via PEP 484 and has since become a standard practice to improve code readability and facilitate static analysis tools. + +**Benefits of Type Hinting** + +1. Improved Readability: Type hints make it clear what type of data is expected, making the code easier to understand for others and your future self. +2. Error Detection: Static analysis tools like MyPy can use type hints to detect type errors before runtime, reducing bugs and improving code quality. +3.Better Tooling Support: Modern IDEs and editors can leverage type hints to provide better autocompletion, refactoring, and error checking features. +4. Documentation: Type hints serve as a form of documentation, indicating the intended usage of functions and classes. + +**Syntax of Type Hinting**
+Type hints can be added to variables, function arguments, and return values using annotations. + +1. Variable Annotations: + +```bash +age: int = 25 +name: str = "Alice" +is_student: bool = True +``` + +2. Function Annotations: + +```bash +def greet(name: str) -> str: + return f"Hello, {name}!" +``` + +3. Multiple Arguments and Return Types: + +```bash +def add(a: int, b: int) -> int: + return a + b +``` + +4. Optional Types: Use the Optional type from the typing module for values that could be None. + +```bash +from typing import Optional + +def get_user_name(user_id: int) -> Optional[str]: + # Function logic here + return None # Example return value +``` + +5. Union Types: Use the Union type when a variable can be of multiple types. + +```bash +from typing import Union + +def get_value(key: str) -> Union[int, str]: + # Function logic here + return "value" # Example return value +``` + +6. List and Dictionary Types: Use the List and Dict types from the typing module for collections. + +```bash +from typing import List, Dict + +def process_data(data: List[int]) -> Dict[str, int]: + # Function logic here + return {"sum": sum(data)} # Example return value +``` + +7. Type Aliases: Create type aliases for complex types to make the code more readable. + +```bash +from typing import List, Tuple + +Coordinates = List[Tuple[int, int]] + +def draw_shape(points: Coordinates) -> None: + # Function logic here + pass +``` + +**Example of Type Hinting in a Class**
+Here is a more comprehensive example using type hints in a class: + +```bash +from typing import List + +class Student: + def __init__(self, name: str, age: int, grades: List[int]) -> None: + self.name = name + self.age = age + self.grades = grades + + def average_grade(self) -> float: + return sum(self.grades) / len(self.grades) + + def add_grade(self, grade: int) -> None: + self.grades.append(grade) + +# Example usage +student = Student("Alice", 20, [90, 85, 88]) +print(student.average_grade()) # Output: 87.66666666666667 +student.add_grade(92) +print(student.average_grade()) # Output: 88.75 +``` + +### Conclusion +Type hinting in Python enhances code readability, facilitates error detection through static analysis, and improves tooling support. By adopting +type hinting, you can write clearer and more maintainable code, reducing the likelihood of bugs and making your codebase easier to navigate for yourself and others. diff --git a/contrib/api-development/index.md b/contrib/api-development/index.md index 72789070..8d4dc595 100644 --- a/contrib/api-development/index.md +++ b/contrib/api-development/index.md @@ -1,4 +1,4 @@ # List of sections - [API Methods](api-methods.md) -- [FastAPI](fast-api.md) \ No newline at end of file +- [FastAPI](fast-api.md) diff --git a/contrib/ds-algorithms/avl-trees.md b/contrib/ds-algorithms/avl-trees.md new file mode 100644 index 00000000..b87e82cb --- /dev/null +++ b/contrib/ds-algorithms/avl-trees.md @@ -0,0 +1,185 @@ +# AVL Tree + +In Data Structures and Algorithms, an **AVL Tree** is a self-balancing binary search tree (BST) where the difference between heights of left and right subtrees cannot be more than one for all nodes. It ensures that the tree remains balanced, providing efficient search, insertion, and deletion operations. + +## Points to be Remembered + +- **Balance Factor**: The difference in heights between the left and right subtrees of a node. It should be -1, 0, or +1 for all nodes in an AVL tree. +- **Rotations**: Tree rotations (left, right, left-right, right-left) are used to maintain the balance factor within the allowed range. + +## Real Life Examples of AVL Trees + +- **Databases**: AVL trees can be used to maintain large indexes for database tables, ensuring quick data retrieval. +- **File Systems**: Some file systems use AVL trees to keep track of free and used memory blocks. + +## Applications of AVL Trees + +AVL trees are used in various applications in Computer Science: + +- **Database Indexing** +- **Memory Allocation** +- **Network Routing Algorithms** + +Understanding these applications is essential for Software Development. + +## Operations in AVL Tree + +Key operations include: + +- **INSERT**: Insert a new element into the AVL tree. +- **SEARCH**: Find the position of an element in the AVL tree. +- **DELETE**: Remove an element from the AVL tree. + +## Implementing AVL Tree in Python + +```python +class AVLTreeNode: + def __init__(self, key): + self.key = key + self.left = None + self.right = None + self.height = 1 + +class AVLTree: + def insert(self, root, key): + if not root: + return AVLTreeNode(key) + + if key < root.key: + root.left = self.insert(root.left, key) + else: + root.right = self.insert(root.right, key) + + root.height = 1 + max(self.getHeight(root.left), self.getHeight(root.right)) + balance = self.getBalance(root) + + if balance > 1 and key < root.left.key: + return self.rotateRight(root) + if balance < -1 and key > root.right.key: + return self.rotateLeft(root) + if balance > 1 and key > root.left.key: + root.left = self.rotateLeft(root.left) + return self.rotateRight(root) + if balance < -1 and key < root.right.key: + root.right = self.rotateRight(root.right) + return self.rotateLeft(root) + + return root + + def search(self, root, key): + if not root or root.key == key: + return root + + if key < root.key: + return self.search(root.left, key) + + return self.search(root.right, key) + + def delete(self, root, key): + if not root: + return root + + if key < root.key: + root.left = self.delete(root.left, key) + elif key > root.key: + root.right = self.delete(root.right, key) + else: + if root.left is None: + temp = root.right + root = None + return temp + elif root.right is None: + temp = root.left + root = None + return temp + + temp = self.getMinValueNode(root.right) + root.key = temp.key + root.right = self.delete(root.right, temp.key) + + if root is None: + return root + + root.height = 1 + max(self.getHeight(root.left), self.getHeight(root.right)) + balance = self.getBalance(root) + + if balance > 1 and self.getBalance(root.left) >= 0: + return self.rotateRight(root) + if balance < -1 and self.getBalance(root.right) <= 0: + return self.rotateLeft(root) + if balance > 1 and self.getBalance(root.left) < 0: + root.left = self.rotateLeft(root.left) + return self.rotateRight(root) + if balance < -1 and self.getBalance(root.right) > 0: + root.right = self.rotateRight(root.right) + return self.rotateLeft(root) + + return root + + def rotateLeft(self, z): + y = z.right + T2 = y.left + y.left = z + z.right = T2 + z.height = 1 + max(self.getHeight(z.left), self.getHeight(z.right)) + y.height = 1 + max(self.getHeight(y.left), self.getHeight(y.right)) + return y + + def rotateRight(self, z): + y = z.left + T3 = y.right + y.right = z + z.left = T3 + z.height = 1 + max(self.getHeight(z.left), self.getHeight(z.right)) + y.height = 1 + max(self.getHeight(y.left), self.getHeight(y.right)) + return y + + def getHeight(self, root): + if not root: + return 0 + return root.height + + def getBalance(self, root): + if not root: + return 0 + return self.getHeight(root.left) - self.getHeight(root.right) + + def getMinValueNode(self, root): + if root is None or root.left is None: + return root + return self.getMinValueNode(root.left) + + def preOrder(self, root): + if not root: + return + print(root.key, end=' ') + self.preOrder(root.left) + self.preOrder(root.right) + +#Example usage +avl_tree = AVLTree() +root = None + +root = avl_tree.insert(root, 10) +root = avl_tree.insert(root, 20) +root = avl_tree.insert(root, 30) +root = avl_tree.insert(root, 40) +root = avl_tree.insert(root, 50) +root = avl_tree.insert(root, 25) + +print("Preorder traversal of the AVL tree is:") +avl_tree.preOrder(root) +``` + +## Output + +```markdown +Preorder traversal of the AVL tree is: +30 20 10 25 40 50 +``` + +## Complexity Analysis + +- **Insertion**: O(logn). Inserting a node involves traversing the height of the tree, which is logarithmic due to the balancing property. +- **Search**: O(logn). Searching for a node involves traversing the height of the tree. +- **Deletion**: O(logn). Deleting a node involves traversing and potentially rebalancing the tree, maintaining the logarithmic height. \ No newline at end of file diff --git a/contrib/ds-algorithms/binary-tree.md b/contrib/ds-algorithms/binary-tree.md new file mode 100644 index 00000000..03da2cf8 --- /dev/null +++ b/contrib/ds-algorithms/binary-tree.md @@ -0,0 +1,231 @@ +# Binary Tree + +A binary tree is a non-linear data structure in which each node can have atmost two children, known as the left and the right child. It is a heirarchial data structure represented in the following way: + +``` + A...................Level 0 + / \ + B C.................Level 1 + / \ \ + D E G...............Level 2 +``` + +## Basic Terminologies + +- **Root node:** The topmost node in a tree is the root node. The root node does not have any parent. In the above example, **A** is the root node. +- **Parent node:** The predecessor of a node is called the parent of that node. **A** is the parent of **B** and **C**, **B** is the parent of **D** and **E** and **C** is the parent of **G**. +- **Child node:** The successor of a node is called the child of that node. **B** and **C** are children of **A**, **D** and **E** are children of **B** and **G** is the right child of **C**. +- **Leaf node:** Nodes without any children are called the leaf nodes. **D**, **E** and **G** are the leaf nodes. +- **Ancestor node:** Predecessor nodes on the path from the root to that node are called ancestor nodes. **A** and **B** are the ancestors of **E**. +- **Descendant node:** Successor nodes on the path from the root to that node are called descendant nodes. **B** and **E** are descendants of **A**. +- **Sibling node:** Nodes having the same parent are called sibling nodes. **B** and **C** are sibling nodes and so are **D** and **E**. +- **Level (Depth) of a node:** Number of edges in the path from the root to that node is the level of that node. The root node is always at level 0. The depth of root node is the depth of the tree. +- **Height of a node:** Number of edges in the path from that node to the deepest leaf is the height of that node. The height of the root is the height of a tree. Height of node **A** is 2, nodes **B** and **C** is 1 and nodes **D**, **E** and **G** is 0. + +## Types Of Binary Trees + +- **Full Binary Tree:** A binary tree where each node has 0 or 2 children is a full binary tree. +``` + A + / \ + B C + / \ + D E +``` +- **Complete Binary Tree:** A binary tree in which all levels are completely filled except the last level is a complete binary tree. Whenever new nodes are inserted, they are inserted from the left side. +``` + A + / \ + / \ + B C + / \ / + D E F +``` +- **Perfect Binary Tree:** A binary tree in which all nodes are completely filled, i.e., each node has two children is called a perfect binary tree. +``` + A + / \ + / \ + B C + / \ / \ + D E F G +``` +- **Skewed Binary Tree:** A binary tree in which each node has either 0 or 1 child is called a skewed binary tree. It is of two types - left skewed binary tree and right skewed binary tree. +``` + A A + \ / + B B + \ / + C C + Right skewed binary tree Left skewed binary tree +``` +- **Balanced Binary Tree:** A binary tree in which the height difference between the left and right subtree is not more than one and the subtrees are also balanced is a balanced binary tree. +``` + A + / \ + B C + / \ + D E +``` + +## Real Life Applications Of Binary Tree + +- **File Systems:** File systems employ binary trees to organize the folders and files, facilitating efficient search and access of files. +- **Decision Trees:** Decision tree, a supervised learning algorithm, utilizes binary trees, with each node representing a decision and its edges showing the possible outcomes. +- **Routing Algorithms:** In routing algorithms, binary trees are used to efficiently transfer data packets from the source to destination through a network of nodes. +- **Searching and sorting Algorithms:** Searching algorithms like binary search and sorting algorithms like heapsort heavily rely on binary trees. + +## Implementation of Binary Tree + +```python +from collections import deque + +class Node: + def __init__(self, data): + self.data = data + self.left = None + self.right = None + +class Binary_tree: + @staticmethod + def insert(root, data): + if root is None: + return Node(data) + q = deque() + q.append(root) + while q: + temp = q.popleft() + if temp.left is None: + temp.left = Node(data) + break + else: + q.append(temp.left) + if temp.right is None: + temp.right = Node(data) + break + else: + q.append(temp.right) + return root + + @staticmethod + def inorder(root): + if not root: + return + b.inorder(root.left) + print(root.data, end=" ") + b.inorder(root.right) + + @staticmethod + def preorder(root): + if not root: + return + print(root.data, end=" ") + b.preorder(root.left) + b.preorder(root.right) + + @staticmethod + def postorder(root): + if not root: + return + b.postorder(root.left) + b.postorder(root.right) + print(root.data, end=" ") + + @staticmethod + def levelorder(root): + if not root: + return + q = deque() + q.append(root) + while q: + temp = q.popleft() + print(temp.data, end=" ") + if temp.left is not None: + q.append(temp.left) + if temp.right is not None: + q.append(temp.right) + + @staticmethod + def delete(root, value): + q = deque() + q.append(root) + while q: + temp = q.popleft() + if temp is value: + temp = None + return + if temp.right: + if temp.right is value: + temp.right = None + return + else: + q.append(temp.right) + if temp.left: + if temp.left is value: + temp.left = None + return + else: + q.append(temp.left) + + @staticmethod + def delete_value(root, value): + if root is None: + return None + if root.left is None and root.right is None: + if root.data == value: + return None + else: + return root + x = None + q = deque() + q.append(root) + temp = None + while q: + temp = q.popleft() + if temp.data == value: + x = temp + if temp.left: + q.append(temp.left) + if temp.right: + q.append(temp.right) + if x: + y = temp.data + x.data = y + b.delete(root, temp) + return root + +b = Binary_tree() +root = None +root = b.insert(root, 10) +root = b.insert(root, 20) +root = b.insert(root, 30) +root = b.insert(root, 40) +root = b.insert(root, 50) +root = b.insert(root, 60) + +print("Preorder traversal:", end=" ") +b.preorder(root) + +print("\nInorder traversal:", end=" ") +b.inorder(root) + +print("\nPostorder traversal:", end=" ") +b.postorder(root) + +print("\nLevel order traversal:", end=" ") +b.levelorder(root) + +root = b.delete_value(root, 20) +print("\nLevel order traversal after deletion:", end=" ") +b.levelorder(root) +``` + +#### OUTPUT + +``` +Preorder traversal: 10 20 40 50 30 60 +Inorder traversal: 40 20 50 10 60 30 +Postorder traversal: 40 50 20 60 30 10 +Level order traversal: 10 20 30 40 50 60 +Level order traversal after deletion: 10 60 30 40 50 +``` diff --git a/contrib/ds-algorithms/deque.md b/contrib/ds-algorithms/deque.md new file mode 100644 index 00000000..2a5a77d2 --- /dev/null +++ b/contrib/ds-algorithms/deque.md @@ -0,0 +1,216 @@ +# Deque in Python + +## Definition +A deque, short for double-ended queue, is an ordered collection of items that allows rapid insertion and deletion at both ends. + +## Syntax +In Python, deques are implemented in the collections module: + +```py +from collections import deque + +# Creating a deque +d = deque(iterable) # Create deque from iterable (optional) +``` + +## Operations +1. **Appending Elements**: + + - append(x): Adds element x to the right end of the deque. + - appendleft(x): Adds element x to the left end of the deque. + + ### Program + ```py + from collections import deque + + # Initialize a deque + d = deque([1, 2, 3, 4, 5]) + print("Initial deque:", d) + + # Append elements + d.append(6) + print("After append(6):", d) + + # Append left + d.appendleft(0) + print("After appendleft(0):", d) + + ``` + ### Output + ```py + Initial deque: deque([1, 2, 3, 4, 5]) + After append(6): deque([1, 2, 3, 4, 5, 6]) + After appendleft(0): deque([0, 1, 2, 3, 4, 5, 6]) + ``` + +2. **Removing Elements**: + + - pop(): Removes and returns the rightmost element. + - popleft(): Removes and returns the leftmost element. + + ### Program + ```py + from collections import deque + + # Initialize a deque + d = deque([1, 2, 3, 4, 5]) + print("Initial deque:", d) + + # Pop from the right end + rightmost = d.pop() + print("Popped from right end:", rightmost) + print("Deque after pop():", d) + + # Pop from the left end + leftmost = d.popleft() + print("Popped from left end:", leftmost) + print("Deque after popleft():", d) + + ``` + + ### Output + ```py + Initial deque: deque([1, 2, 3, 4, 5]) + Popped from right end: 5 + Deque after pop(): deque([1, 2, 3, 4]) + Popped from left end: 1 + Deque after popleft(): deque([2, 3, 4]) + ``` + +3. **Accessing Elements**: + + - deque[index]: Accesses element at index. + + ### Program + ```py + from collections import deque + + # Initialize a deque + d = deque([1, 2, 3, 4, 5]) + print("Initial deque:", d) + + # Accessing elements + print("Element at index 2:", d[2]) + + ``` + + ### Output + ```py + Initial deque: deque([1, 2, 3, 4, 5]) + Element at index 2: 3 + + ``` + +4. **Other Operations**: + + - extend(iterable): Extends deque by appending elements from iterable. + - extendleft(iterable): Extends deque by appending elements from iterable to the left. + - rotate(n): Rotates deque n steps to the right (negative n rotates left). + + ### Program + ```py + from collections import deque + + # Initialize a deque + d = deque([1, 2, 3, 4, 5]) + print("Initial deque:", d) + + # Extend deque + d.extend([6, 7, 8]) + print("After extend([6, 7, 8]):", d) + + # Extend left + d.extendleft([-1, 0]) + print("After extendleft([-1, 0]):", d) + + # Rotate deque + d.rotate(2) + print("After rotate(2):", d) + + # Rotate left + d.rotate(-3) + print("After rotate(-3):", d) + + ``` + + ### Output + ```py + Initial deque: deque([1, 2, 3, 4, 5]) + After extend([6, 7, 8]): deque([1, 2, 3, 4, 5, 6, 7, 8]) + After extendleft([-1, 0]): deque([0, -1, 1, 2, 3, 4, 5, 6, 7, 8]) + After rotate(2): deque([7, 8, 0, -1, 1, 2, 3, 4, 5, 6]) + After rotate(-3): deque([1, 2, 3, 4, 5, 6, 7, 8, 0, -1]) + + ``` + + +## Example + +### 1. Finding Maximum in Sliding Window +```py +from collections import deque + +def max_sliding_window(nums, k): + if not nums: + return [] + + d = deque() + result = [] + + for i, num in enumerate(nums): + # Remove elements from deque that are out of the current window + if d and d[0] <= i - k: + d.popleft() + + # Remove elements from deque smaller than the current element + while d and nums[d[-1]] <= num: + d.pop() + + d.append(i) + + # Add maximum for current window + if i >= k - 1: + result.append(nums[d[0]]) + + return result + +# Example usage: +nums = [1, 3, -1, -3, 5, 3, 6, 7] +k = 3 +print("Maximums in sliding window of size", k, "are:", max_sliding_window(nums, k)) + +``` + +Output +```py +Maximums in sliding window of size 3 are: [3, 3, 5, 5, 6, 7] +``` + + +## Applications +- **Efficient Queues and Stacks**: Deques allow fast O(1) append and pop operations from both ends, +making them ideal for implementing queues and stacks. +- **Sliding Window Maximum/Minimum**: Used in algorithms that require efficient windowed +computations. + + +## Advantages +- Efficiency: O(1) time complexity for append and pop operations from both ends. +- Versatility: Can function both as a queue and as a stack. +- Flexible: Supports rotation and slicing operations efficiently. + + +## Disadvantages +- Memory Usage: Requires more memory compared to simple lists due to overhead in managing linked +nodes. + +## Conclusion +- Deques in Python, provided by the collections.deque module, offer efficient double-ended queue +operations with O(1) time complexity for append and pop operations on both ends. They are versatile +data structures suitable for implementing queues, stacks, and more complex algorithms requiring +efficient manipulation of elements at both ends. + +- While deques excel in scenarios requiring fast append and pop operations from either end, they do +consume more memory compared to simple lists due to their implementation using doubly-linked lists. +However, their flexibility and efficiency make them invaluable for various programming tasks and +algorithmic solutions. \ No newline at end of file diff --git a/contrib/ds-algorithms/dijkstra.md b/contrib/ds-algorithms/dijkstra.md new file mode 100644 index 00000000..cea6da40 --- /dev/null +++ b/contrib/ds-algorithms/dijkstra.md @@ -0,0 +1,90 @@ + +# Dijkstra's Algorithm +Dijkstra's algorithm is a graph algorithm that gives the shortest distance of each node from the given node in a weighted, undirected graph. It operates by continually choosing the closest unvisited node and determining the distance to all its unvisited neighboring nodes. This algorithm is similar to BFS in graphs, with the difference being it gives priority to nodes with shorter distances by using a priority queue(min-heap) instead of a FIFO queue. The data structures required would be a distance list (to store the minimum distance of each node), a priority queue or a set, and we assume the adjacency list will be provided. + +## Working +- We will store the minimum distance of each node in the distance list, which has a length equal to the number of nodes in the graph. Thus, the minimum distance of the 2nd node will be stored in the 2nd index of the distance list. We initialize the list with the maximum number possible, say infinity. + +- We now start the traversal from the starting node given and mark its distance as 0. We push this node to the priority queue along with its minimum distance, which is 0, so the structure pushed will be (0, node), a tuple. + +- Now, with the help of the adjacency list, we will add the neighboring nodes to the priority queue with the distance equal to (edge weight + current node distance), and this should be less than the distance list value. We will also update the distance list in the process. + +- When all the nodes are added, we will select the node with the shortest distance and repeat the process. + +## Dry Run +We will now do a manual simulation using an example graph given. First, (0, a) is pushed to the priority queue (pq). + + +- **Step1:** The lowest element is popped from the pq, which is (0, a), and all its neighboring nodes are added to the pq while simultaneously checking the distance list. Thus (3, b), (7, c), (1, d) are added to the pq. + + +- **Step2:** Again, the lowest element is popped from the pq, which is (1, d). It has two neighboring nodes, a and e, from which + (0 + 1, a) will not be added to the pq as dist[a] = 0 is less than 1. + + +- **Step3:** Now, the lowest element is popped from the pq, which is (3, b). It has two neighboring nodes, a and c, from which + (0 + 1, a) will not be added to the pq. But the new distance to reach c is 5 (3 + 2), which is less than dist[c] = 7. So (5, c) is added to the pq. + + +- **Step4:** The next smallest element is (5, c). It has neighbors a and e. The new distance to reach a will be 5 + 7 = 12, which is more than dist[a], so it will not be considered. Similarly, the new distance for e is 5 + 3 = 8, which again will not be considered. So, no new tuple has been added to the pq. + + +- **Step5:** Similarly, both the elements of the pq will be popped one by one without any new addition. + + + +- The distance list we get at the end will be our answer. +- `Output` `dist=[1, 3, 7, 1, 6]` + +## Python Code +```python +import heapq + +def dijkstra(graph, start): + # Create a priority queue + pq = [] + heapq.heappush(pq, (0, start)) + + # Create a dictionary to store distances to each node + dist = {node: float('inf') for node in graph} + dist[start] = 0 + + while pq: + # Get the node with the smallest distance + current_distance, current_node = heapq.heappop(pq) + + # If the current distance is greater than the recorded distance, skip it + if current_distance > dist[current_node]: + continue + + # Update the distances to the neighboring nodes + for neighbor, weight in graph[current_node].items(): + distance = current_distance + weight + # Only consider this new path if it's better + if distance < dist[neighbor]: + dist[neighbor] = distance + heapq.heappush(pq, (distance, neighbor)) + + return dist + +# Example usage: +graph = { + 'A': {'B': 1, 'C': 4}, + 'B': {'A': 1, 'C': 2, 'D': 5}, + 'C': {'A': 4, 'B': 2, 'D': 1}, + 'D': {'B': 5, 'C': 1} +} + +start_node = 'A' +dist = dijkstra(graph, start_node) +print(dist) +``` + +## Complexity Analysis + +- **Time Complexity**: \(O((V + E) log V)\) +- **Space Complexity**: \(O(V + E)\) + + + + diff --git a/contrib/ds-algorithms/dynamic-programming.md b/contrib/ds-algorithms/dynamic-programming.md index 43149f86..f4958689 100644 --- a/contrib/ds-algorithms/dynamic-programming.md +++ b/contrib/ds-algorithms/dynamic-programming.md @@ -51,10 +51,6 @@ print(f"The {n}th Fibonacci number is: {fibonacci(n)}.") - **Time Complexity**: O(n) for both approaches - **Space Complexity**: O(n) for the top-down approach (due to memoization), O(1) for the bottom-up approach - -
- - # 2. Longest Common Subsequence The longest common subsequence (LCS) problem is to find the longest subsequence common to two sequences. A subsequence is a sequence that appears in the same relative order but not necessarily contiguous. @@ -84,14 +80,34 @@ Y = "GXTXAYB" print("Length of Longest Common Subsequence:", longest_common_subsequence(X, Y, len(X), len(Y))) ``` +## Longest Common Subsequence Code in Python (Bottom-Up Approach) + +```python + +def longestCommonSubsequence(X, Y, m, n): + L = [[None]*(n+1) for i in range(m+1)] + for i in range(m+1): + for j in range(n+1): + if i == 0 or j == 0: + L[i][j] = 0 + elif X[i-1] == Y[j-1]: + L[i][j] = L[i-1][j-1]+1 + else: + L[i][j] = max(L[i-1][j], L[i][j-1]) + return L[m][n] + + +S1 = "AGGTAB" +S2 = "GXTXAYB" +m = len(S1) +n = len(S2) +print("Length of LCS is", longestCommonSubsequence(S1, S2, m, n)) +``` + ## Complexity Analysis -- **Time Complexity**: O(m * n) for the top-down approach, where m and n are the lengths of the input sequences +- **Time Complexity**: O(m * n) for both approaches, where m and n are the lengths of the input sequences - **Space Complexity**: O(m * n) for the memoization table - -
- - # 3. 0-1 Knapsack Problem The 0-1 knapsack problem is a classic optimization problem where the goal is to maximize the total value of items selected while keeping the total weight within a specified limit. @@ -123,10 +139,315 @@ n = len(weights) print("Maximum value that can be obtained:", knapsack(weights, values, capacity, n)) ``` +## 0-1 Knapsack Problem Code in Python (Bottom-up Approach) + +```python +def knapSack(capacity, weights, values, n): + K = [[0 for x in range(capacity + 1)] for x in range(n + 1)] + for i in range(n + 1): + for w in range(capacity + 1): + if i == 0 or w == 0: + K[i][w] = 0 + elif weights[i-1] <= w: + K[i][w] = max(values[i-1] + + K[i-1][w-weights[i-1]], + K[i-1][w]) + else: + K[i][w] = K[i-1][w] + + return K[n][capacity] + +values = [60, 100, 120] +weights = [10, 20, 30] +capacity = 50 +n = len(weights) +print(knapSack(capacity, weights, values, n)) +``` + ## Complexity Analysis -- **Time Complexity**: O(n * W) for the top-down approach, where n is the number of items and W is the capacity of the knapsack +- **Time Complexity**: O(n * W) for both approaches, where n is the number of items and W is the capacity of the knapsack - **Space Complexity**: O(n * W) for the memoization table - -
- \ No newline at end of file +# 4. Longest Increasing Subsequence + +The Longest Increasing Subsequence (LIS) is a task is to find the longest subsequence that is strictly increasing, meaning each element in the subsequence is greater than the one before it. This subsequence must maintain the order of elements as they appear in the original sequence but does not need to be contiguous. The goal is to identify the subsequence with the maximum possible length. + +**Algorithm Overview:** +- **Base cases:** If the sequence is empty, the LIS length is 0. +- **Memoization:** Store the results of previously computed subproblems to avoid redundant computations. +- **Recurrence relation:** Compute the LIS length by comparing characters of the sequences and making decisions based on their values. + +## Longest Increasing Subsequence Code in Python (Top-Down Approach using Memoization) + +```python +import sys + +def f(idx, prev_idx, n, a, dp): + if (idx == n): + return 0 + + if (dp[idx][prev_idx + 1] != -1): + return dp[idx][prev_idx + 1] + + notTake = 0 + f(idx + 1, prev_idx, n, a, dp) + take = -sys.maxsize - 1 + if (prev_idx == -1 or a[idx] > a[prev_idx]): + take = 1 + f(idx + 1, idx, n, a, dp) + + dp[idx][prev_idx + 1] = max(take, notTake) + return dp[idx][prev_idx + 1] + +def longestSubsequence(n, a): + + dp = [[-1 for i in range(n + 1)]for j in range(n + 1)] + return f(0, -1, n, a, dp) + +a = [3, 10, 2, 1, 20] +n = len(a) + +print("Length of lis is", longestSubsequence(n, a)) + +``` + +## Longest Increasing Subsequence Code in Python (Bottom-Up Approach) + +```python +def lis(arr): + n = len(arr) + lis = [1]*n + + for i in range(1, n): + for j in range(0, i): + if arr[i] > arr[j] and lis[i] < lis[j] + 1: + lis[i] = lis[j]+1 + + maximum = 0 + for i in range(n): + maximum = max(maximum, lis[i]) + + return maximum + +arr = [10, 22, 9, 33, 21, 50, 41, 60] +print("Length of lis is", lis(arr)) +``` + +## Complexity Analysis +- **Time Complexity**: O(n * n) for both approaches, where n is the length of the array. +- **Space Complexity**: O(n * n) for the memoization table in Top-Down Approach, O(n) in Bottom-Up Approach. + +# 5. String Edit Distance + +The String Edit Distance algorithm calculates the minimum number of operations (insertions, deletions, or substitutions) required to convert one string into another. + +**Algorithm Overview:** +- **Base Cases:** If one string is empty, the edit distance is the length of the other string. +- **Memoization:** Store the results of previously computed edit distances to avoid redundant computations. +- **Recurrence Relation:** Compute the edit distance by considering insertion, deletion, and substitution operations. + +## String Edit Distance Code in Python (Top-Down Approach with Memoization) +```python +def edit_distance(str1, str2, memo={}): + m, n = len(str1), len(str2) + if (m, n) in memo: + return memo[(m, n)] + if m == 0: + return n + if n == 0: + return m + if str1[m - 1] == str2[n - 1]: + memo[(m, n)] = edit_distance(str1[:m-1], str2[:n-1], memo) + else: + memo[(m, n)] = 1 + min(edit_distance(str1, str2[:n-1], memo), # Insert + edit_distance(str1[:m-1], str2, memo), # Remove + edit_distance(str1[:m-1], str2[:n-1], memo)) # Replace + return memo[(m, n)] + +str1 = "sunday" +str2 = "saturday" +print(f"Edit Distance between '{str1}' and '{str2}' is {edit_distance(str1, str2)}.") +``` + +#### Output +``` +Edit Distance between 'sunday' and 'saturday' is 3. +``` + +## String Edit Distance Code in Python (Bottom-Up Approach) +```python +def edit_distance(str1, str2): + m, n = len(str1), len(str2) + dp = [[0 for _ in range(n + 1)] for _ in range(m + 1)] + + for i in range(m + 1): + for j in range(n + 1): + if i == 0: + dp[i][j] = j + elif j == 0: + dp[i][j] = i + elif str1[i - 1] == str2[j - 1]: + dp[i][j] = dp[i - 1][j - 1] + else: + dp[i][j] = 1 + min(dp[i - 1][j], dp[i][j - 1], dp[i - 1][j - 1]) + + return dp[m][n] + +str1 = "sunday" +str2 = "saturday" +print(f"Edit Distance between '{str1}' and '{str2}' is {edit_distance(str1, str2)}.") +``` + +#### Output +``` +Edit Distance between 'sunday' and 'saturday' is 3. +``` + +## **Complexity Analysis:** +- **Time Complexity:** O(m * n) where m and n are the lengths of string 1 and string 2 respectively +- **Space Complexity:** O(m * n) for both top-down and bottom-up approaches + + +# 6. Matrix Chain Multiplication + +The Matrix Chain Multiplication finds the optimal way to multiply a sequence of matrices to minimize the number of scalar multiplications. + +**Algorithm Overview:** +- **Base Cases:** The cost of multiplying one matrix is zero. +- **Memoization:** Store the results of previously computed matrix chain orders to avoid redundant computations. +- **Recurrence Relation:** Compute the optimal cost by splitting the product at different points and choosing the minimum cost. + +## Matrix Chain Multiplication Code in Python (Top-Down Approach with Memoization) +```python +def matrix_chain_order(p, memo={}): + n = len(p) - 1 + def compute_cost(i, j): + if (i, j) in memo: + return memo[(i, j)] + if i == j: + return 0 + memo[(i, j)] = float('inf') + for k in range(i, j): + q = compute_cost(i, k) + compute_cost(k + 1, j) + p[i - 1] * p[k] * p[j] + if q < memo[(i, j)]: + memo[(i, j)] = q + return memo[(i, j)] + return compute_cost(1, n) + +p = [1, 2, 3, 4] +print(f"Minimum number of multiplications is {matrix_chain_order(p)}.") +``` + +#### Output +``` +Minimum number of multiplications is 18. +``` + + +## Matrix Chain Multiplication Code in Python (Bottom-Up Approach) +```python +def matrix_chain_order(p): + n = len(p) - 1 + m = [[0 for _ in range(n)] for _ in range(n)] + + for L in range(2, n + 1): + for i in range(n - L + 1): + j = i + L - 1 + m[i][j] = float('inf') + for k in range(i, j): + q = m[i][k] + m[k + 1][j] + p[i] * p[k + 1] * p[j + 1] + if q < m[i][j]: + m[i][j] = q + + return m[0][n - 1] + +p = [1, 2, 3, 4] +print(f"Minimum number of multiplications is {matrix_chain_order(p)}.") +``` + +#### Output +``` +Minimum number of multiplications is 18. +``` + +## **Complexity Analysis:** +- **Time Complexity:** O(n^3) where n is the number of matrices in the chain. For an `array p` of dimensions representing the matrices such that the `i-th matrix` has dimensions `p[i-1] x p[i]`, n is `len(p) - 1` +- **Space Complexity:** O(n^2) for both top-down and bottom-up approaches + +# 7. Optimal Binary Search Tree + +The Matrix Chain Multiplication finds the optimal way to multiply a sequence of matrices to minimize the number of scalar multiplications. + +**Algorithm Overview:** +- **Base Cases:** The cost of a single key is its frequency. +- **Memoization:** Store the results of previously computed subproblems to avoid redundant computations. +- **Recurrence Relation:** Compute the optimal cost by trying each key as the root and choosing the minimum cost. + +## Optimal Binary Search Tree Code in Python (Top-Down Approach with Memoization) + +```python +def optimal_bst(keys, freq, memo={}): + n = len(keys) + def compute_cost(i, j): + if (i, j) in memo: + return memo[(i, j)] + if i > j: + return 0 + if i == j: + return freq[i] + memo[(i, j)] = float('inf') + total_freq = sum(freq[i:j+1]) + for r in range(i, j + 1): + cost = (compute_cost(i, r - 1) + + compute_cost(r + 1, j) + + total_freq) + if cost < memo[(i, j)]: + memo[(i, j)] = cost + return memo[(i, j)] + return compute_cost(0, n - 1) + +keys = [10, 12, 20] +freq = [34, 8, 50] +print(f"Cost of Optimal BST is {optimal_bst(keys, freq)}.") +``` + +#### Output +``` +Cost of Optimal BST is 142. +``` + +## Optimal Binary Search Tree Code in Python (Bottom-Up Approach) + +```python +def optimal_bst(keys, freq): + n = len(keys) + cost = [[0 for x in range(n)] for y in range(n)] + + for i in range(n): + cost[i][i] = freq[i] + + for L in range(2, n + 1): + for i in range(n - L + 1): + j = i + L - 1 + cost[i][j] = float('inf') + total_freq = sum(freq[i:j+1]) + for r in range(i, j + 1): + c = (cost[i][r - 1] if r > i else 0) + \ + (cost[r + 1][j] if r < j else 0) + \ + total_freq + if c < cost[i][j]: + cost[i][j] = c + + return cost[0][n - 1] + +keys = [10, 12, 20] +freq = [34, 8, 50] +print(f"Cost of Optimal BST is {optimal_bst(keys, freq)}.") +``` + +#### Output +``` +Cost of Optimal BST is 142. +``` + +### Complexity Analysis +- **Time Complexity**: O(n^3) where n is the number of keys in the binary search tree. +- **Space Complexity**: O(n^2) for both top-down and bottom-up approaches diff --git a/contrib/ds-algorithms/hash-tables.md b/contrib/ds-algorithms/hash-tables.md new file mode 100644 index 00000000..f03b7c5e --- /dev/null +++ b/contrib/ds-algorithms/hash-tables.md @@ -0,0 +1,212 @@ +# Data Structures: Hash Tables, Hash Sets, and Hash Maps + +## Table of Contents +- [Introduction](#introduction) +- [Hash Tables](#hash-tables) + - [Overview](#overview) + - [Operations](#operations) +- [Hash Sets](#hash-sets) + - [Overview](#overview-1) + - [Operations](#operations-1) +- [Hash Maps](#hash-maps) + - [Overview](#overview-2) + - [Operations](#operations-2) +- [Conclusion](#conclusion) + +## Introduction +This document provides an overview of three fundamental data structures in computer science: hash tables, hash sets, and hash maps. These structures are widely used for efficient data storage and retrieval operations. + +## Hash Tables + +### Overview +A **hash table** is a data structure that stores key-value pairs. It uses a hash function to compute an index into an array of buckets or slots, from which the desired value can be found. + +### Operations +1. **Insertion**: Add a new key-value pair to the hash table. +2. **Deletion**: Remove a key-value pair from the hash table. +3. **Search**: Find the value associated with a given key. +4. **Update**: Modify the value associated with a given key. + +**Example Code (Python):** +```python +class Node: + def __init__(self, key, value): + self.key = key + self.value = value + self.next = None + + +class HashTable: + def __init__(self, capacity): + self.capacity = capacity + self.size = 0 + self.table = [None] * capacity + + def _hash(self, key): + return hash(key) % self.capacity + + def insert(self, key, value): + index = self._hash(key) + + if self.table[index] is None: + self.table[index] = Node(key, value) + self.size += 1 + else: + current = self.table[index] + while current: + if current.key == key: + current.value = value + return + current = current.next + new_node = Node(key, value) + new_node.next = self.table[index] + self.table[index] = new_node + self.size += 1 + + def search(self, key): + index = self._hash(key) + + current = self.table[index] + while current: + if current.key == key: + return current.value + current = current.next + + raise KeyError(key) + + def remove(self, key): + index = self._hash(key) + + previous = None + current = self.table[index] + + while current: + if current.key == key: + if previous: + previous.next = current.next + else: + self.table[index] = current.next + self.size -= 1 + return + previous = current + current = current.next + + raise KeyError(key) + + def __len__(self): + return self.size + + def __contains__(self, key): + try: + self.search(key) + return True + except KeyError: + return False + + +# Driver code +if __name__ == '__main__': + + ht = HashTable(5) + + ht.insert("apple", 3) + ht.insert("banana", 2) + ht.insert("cherry", 5) + + + print("apple" in ht) + print("durian" in ht) + + print(ht.search("banana")) + + ht.insert("banana", 4) + print(ht.search("banana")) # 4 + + ht.remove("apple") + + print(len(ht)) # 3 +``` + +# Insert elements +hash_table["key1"] = "value1" +hash_table["key2"] = "value2" + +# Search for an element +value = hash_table.get("key1") + +# Delete an element +del hash_table["key2"] + +# Update an element +hash_table["key1"] = "new_value1" + +## Hash Sets + +### Overview +A **hash set** is a collection of unique elements. It is implemented using a hash table where each bucket can store only one element. + +### Operations +1. **Insertion**: Add a new element to the set. +2. **Deletion**: Remove an element from the set. +3. **Search**: Check if an element exists in the set. +4. **Union**: Combine two sets to form a new set with elements from both. +5. **Intersection**: Find common elements between two sets. +6. **Difference**: Find elements present in one set but not in the other. + +**Example Code (Python):** +```python +# Create a hash set +hash_set = set() + +# Insert elements +hash_set.add("element1") +hash_set.add("element2") + +# Search for an element +exists = "element1" in hash_set + +# Delete an element +hash_set.remove("element2") + +# Union of sets +another_set = {"element3", "element4"} +union_set = hash_set.union(another_set) + +# Intersection of sets +intersection_set = hash_set.intersection(another_set) + +# Difference of sets +difference_set = hash_set.difference(another_set) +``` +## Hash Maps + +### Overview +A **hash map** is similar to a hash table but often provides additional functionalities and more user-friendly interfaces for developers. It is a collection of key-value pairs where each key is unique. + +### Operations +1. **Insertion**: Add a new key-value pair to the hash map. +2. **Deletion**: Remove a key-value pair from the hash map. +3. **Search**: Retrieve the value associated with a given key. +4. **Update**: Change the value associated with a given key. + +**Example Code (Python):** +```python +# Create a hash map +hash_map = {} + +# Insert elements +hash_map["key1"] = "value1" +hash_map["key2"] = "value2" + +# Search for an element +value = hash_map.get("key1") + +# Delete an element +del hash_map["key2"] + +# Update an element +hash_map["key1"] = "new_value1" + +``` +## Conclusion +Hash tables, hash sets, and hash maps are powerful data structures that provide efficient means of storing and retrieving data. Understanding these structures and their operations is crucial for developing optimized algorithms and applications. \ No newline at end of file diff --git a/contrib/ds-algorithms/hashing-chaining.md b/contrib/ds-algorithms/hashing-chaining.md new file mode 100644 index 00000000..34086b5e --- /dev/null +++ b/contrib/ds-algorithms/hashing-chaining.md @@ -0,0 +1,153 @@ +# Hashing with Chaining + +In Data Structures and Algorithms, hashing is used to map data of arbitrary size to fixed-size values. A common approach to handle collisions in hashing is **chaining**. In chaining, each slot of the hash table contains a linked list, and all elements that hash to the same slot are stored in that list. + +## Points to be Remembered + +- **Hash Function**: A function that converts an input (or 'key') into an index in a hash table. +- **Collision**: When two keys hash to the same index. +- **Chaining**: A method to resolve collisions by maintaining a linked list for each hash table slot. + +## Real Life Examples of Hashing with Chaining + +- **Phone Directory**: Contacts are stored in a hash table where the contact's name is hashed to an index. If multiple names hash to the same index, they are stored in a linked list at that index. +- **Library Catalog**: Books are indexed by their titles. If multiple books have titles that hash to the same index, they are stored in a linked list at that index. + +## Applications of Hashing + +Hashing is widely used in Computer Science: + +- **Database Indexing** +- **Caches** (like CPU caches, web caches) +- **Associative Arrays** (or dictionaries in Python) +- **Sets** (unordered collections of unique elements) + +Understanding these applications is essential for Software Development. + +## Operations in Hash Table with Chaining + +Key operations include: + +- **INSERT**: Insert a new element into the hash table. +- **SEARCH**: Find the position of an element in the hash table. +- **DELETE**: Remove an element from the hash table. + +## Implementing Hash Table with Chaining in Python + +```python +class Node: + def __init__(self, key, value): + self.key = key + self.value = value + self.next = None + +class HashTable: + def __init__(self, size): + self.size = size + self.table = [None] * size + + def hash_function(self, key): + return key % self.size + + def insert(self, key, value): + hash_index = self.hash_function(key) + new_node = Node(key, value) + + if self.table[hash_index] is None: + self.table[hash_index] = new_node + else: + current = self.table[hash_index] + while current.next is not None: + current = current.next + current.next = new_node + + def search(self, key): + hash_index = self.hash_function(key) + current = self.table[hash_index] + + while current is not None: + if current.key == key: + return current.value + current = current.next + + return None + + def delete(self, key): + hash_index = self.hash_function(key) + current = self.table[hash_index] + prev = None + + while current is not None: + if current.key == key: + if prev is None: + self.table[hash_index] = current.next + else: + prev.next = current.next + return True + prev = current + current = current.next + + return False + + def display(self): + for index, item in enumerate(self.table): + print(f"Index {index}:", end=" ") + current = item + while current is not None: + print(f"({current.key}, {current.value})", end=" -> ") + current = current.next + print("None") + +# Example usage +hash_table = HashTable(10) + +hash_table.insert(1, 'A') +hash_table.insert(11, 'B') +hash_table.insert(21, 'C') + +print("Hash Table after Insert operations:") +hash_table.display() + +print("Search operation for key 11:", hash_table.search(11)) + +hash_table.delete(11) + +print("Hash Table after Delete operation:") +hash_table.display() +``` + +## Output + +```markdown +Hash Table after Insert operations: +Index 0: None +Index 1: (1, 'A') -> (11, 'B') -> (21, 'C') -> None +Index 2: None +Index 3: None +Index 4: None +Index 5: None +Index 6: None +Index 7: None +Index 8: None +Index 9: None + +Search operation for key 11: B + +Hash Table after Delete operation: +Index 0: None +Index 1: (1, 'A') -> (21, 'C') -> None +Index 2: None +Index 3: None +Index 4: None +Index 5: None +Index 6: None +Index 7: None +Index 8: None +Index 9: None +``` + +## Complexity Analysis + +- **Insertion**: Average case O(1), Worst case O(n) when many elements hash to the same slot. +- **Search**: Average case O(1), Worst case O(n) when many elements hash to the same slot. +- **Deletion**: Average case O(1), Worst case O(n) when many elements hash to the same slot. \ No newline at end of file diff --git a/contrib/ds-algorithms/hashing-linear-probing.md b/contrib/ds-algorithms/hashing-linear-probing.md new file mode 100644 index 00000000..0d27db47 --- /dev/null +++ b/contrib/ds-algorithms/hashing-linear-probing.md @@ -0,0 +1,139 @@ +# Hashing with Linear Probing + +In Data Structures and Algorithms, hashing is used to map data of arbitrary size to fixed-size values. A common approach to handle collisions in hashing is **linear probing**. In linear probing, if a collision occurs (i.e., the hash value points to an already occupied slot), we linearly probe through the table to find the next available slot. This method ensures that every element can be inserted or found in the hash table. + +## Points to be Remembered + +- **Hash Function**: A function that converts an input (or 'key') into an index in a hash table. +- **Collision**: When two keys hash to the same index. +- **Linear Probing**: A method to resolve collisions by checking the next slot (i.e., index + 1) until an empty slot is found. + +## Real Life Examples of Hashing with Linear Probing + +- **Student Record System**: Each student record is stored in a table where the student's ID number is hashed to an index. If two students have the same hash index, linear probing finds the next available slot. +- **Library System**: Books are indexed by their ISBN numbers. If two books hash to the same slot, linear probing helps find another spot for the book in the catalog. + +## Applications of Hashing + +Hashing is widely used in Computer Science: + +- **Database Indexing** +- **Caches** (like CPU caches, web caches) +- **Associative Arrays** (or dictionaries in Python) +- **Sets** (unordered collections of unique elements) + +Understanding these applications is essential for Software Development. + +## Operations in Hash Table with Linear Probing + +Key operations include: + +- **INSERT**: Insert a new element into the hash table. +- **SEARCH**: Find the position of an element in the hash table. +- **DELETE**: Remove an element from the hash table. + +## Implementing Hash Table with Linear Probing in Python + +```python +class HashTable: + def __init__(self, size): + self.size = size + self.table = [None] * size + + def hash_function(self, key): + return key % self.size + + def insert(self, key, value): + hash_index = self.hash_function(key) + + if self.table[hash_index] is None: + self.table[hash_index] = (key, value) + else: + while self.table[hash_index] is not None: + hash_index = (hash_index + 1) % self.size + self.table[hash_index] = (key, value) + + def search(self, key): + hash_index = self.hash_function(key) + + while self.table[hash_index] is not None: + if self.table[hash_index][0] == key: + return self.table[hash_index][1] + hash_index = (hash_index + 1) % self.size + + return None + + def delete(self, key): + hash_index = self.hash_function(key) + + while self.table[hash_index] is not None: + if self.table[hash_index][0] == key: + self.table[hash_index] = None + return True + hash_index = (hash_index + 1) % self.size + + return False + + def display(self): + for index, item in enumerate(self.table): + print(f"Index {index}: {item}") + +# Example usage +hash_table = HashTable(10) + +hash_table.insert(1, 'A') +hash_table.insert(11, 'B') +hash_table.insert(21, 'C') + +print("Hash Table after Insert operations:") +hash_table.display() + +print("Search operation for key 11:", hash_table.search(11)) + +hash_table.delete(11) + +print("Hash Table after Delete operation:") +hash_table.display() +``` + +## Output + +```markdown +Hash Table after Insert operations: +Index 0: None +Index 1: (1, 'A') +Index 2: None +Index 3: None +Index 4: None +Index 5: None +Index 6: None +Index 7: None +Index 8: None +Index 9: None +Index 10: None +Index 11: (11, 'B') +Index 12: (21, 'C') + +Search operation for key 11: B + +Hash Table after Delete operation: +Index 0: None +Index 1: (1, 'A') +Index 2: None +Index 3: None +Index 4: None +Index 5: None +Index 6: None +Index 7: None +Index 8: None +Index 9: None +Index 10: None +Index 11: None +Index 12: (21, 'C') +``` + +## Complexity Analysis + +- **Insertion**: Average case O(1), Worst case O(n) when many collisions occur. +- **Search**: Average case O(1), Worst case O(n) when many collisions occur. +- **Deletion**: Average case O(1), Worst case O(n) when many collisions occur. \ No newline at end of file diff --git a/contrib/ds-algorithms/heaps.md b/contrib/ds-algorithms/heaps.md new file mode 100644 index 00000000..6a9ba71a --- /dev/null +++ b/contrib/ds-algorithms/heaps.md @@ -0,0 +1,169 @@ +# Heaps + +## Definition: +Heaps are a crucial data structure that support efficient priority queue operations. They come in two main types: min heaps and max heaps. Python's heapq module provides a robust implementation for min heaps, and with some minor adjustments, it can also be used to implement max heaps. + +## Overview: +A heap is a specialized binary tree-based data structure that satisfies the heap property: + +- **Min Heap:** The key at the root must be the minimum among all keys present in the Binary Heap. This property must be recursively true for all nodes in the Binary Tree. + +- **Max Heap:** The key at the root must be the maximum among all keys present in the Binary Heap. This property must be recursively true for all nodes in the Binary Tree. + +## Python heapq Module: +The heapq module provides an implementation of the heap queue algorithm, also known as the priority queue algorithm. + +- **Min Heap:** In a min heap, the smallest element is always at the root. Here's how to use heapq to create and manipulate a min heap: + + ```python + import heapq + +# Create an empty heap +min_heap = [] + +# Adding elements to the heap + +heapq.heappush(min_heap, 10) +heapq.heappush(min_heap, 5) +heapq.heappush(min_heap, 3) +heapq.heappush(min_heap, 12) +print("Min Heap:", min_heap) + +# Pop the smallest element +smallest = heapq.heappop(min_heap) +print("Smallest element:", smallest) +print("Min Heap after pop:", min_heap) +``` + +**Output:** + + ``` +Min Heap: [3, 5, 10, 12] +Smallest element: 3 +Min Heap after pop: [5, 12, 10] +``` + +- **Max Heap:** To create a max heap, we can store negative values. + +```python +import heapq + +# Create an empty heap +max_heap = [] + +# Adding elements to the heap by pushing negative values +heapq.heappush(max_heap, -10) +heapq.heappush(max_heap, -5) +heapq.heappush(max_heap, -3) +heapq.heappush(max_heap, -12) + +# Convert back to positive values for display +print("Max Heap:", [-x for x in max_heap]) + +# Pop the largest element +largest = -heapq.heappop(max_heap) +print("Largest element:", largest) +print("Max Heap after pop:", [-x for x in max_heap]) + +``` + +**Output:** + +``` +Max Heap: [12, 10, 3, 5] +Largest element: 12 +Max Heap after pop: [10, 5, 3] +``` + +## Heap Operations: +1. **Push Operation:** Adds an element to the heap, maintaining the heap property. +```python +heapq.heappush(heap, item) +``` +2. **Pop Operation:** Removes and returns the smallest element from the heap. +```python +smallest = heapq.heappop(heap) +``` +3. **Heapify Operation:** Converts a list into a heap in-place. +```python +heapq.heapify(list) +``` +4. **Peek Operation:** To get the smallest element without popping it (not directly available, but can be done by accessing the first element). +```python +smallest = heap[0] +``` + +## Example: +```python +# importing "heapq" to implement heap queue +import heapq + +# initializing list +li = [15, 77, 90, 1, 3] + +# using heapify to convert list into heap +heapq.heapify(li) + +# printing created heap +print("The created heap is : ", end="") +print(list(li)) + +# using heappush() to push elements into heap +# pushes 4 +heapq.heappush(li, 4) + +# printing modified heap +print("The modified heap after push is : ", end="") +print(list(li)) + +# using heappop() to pop smallest element +print("The popped and smallest element is : ", end="") +print(heapq.heappop(li)) + +``` + +Output: +``` +The created heap is : [1, 3, 15, 77, 90] +The modified heap after push is : [1, 3, 4, 15, 77, 90] +The popped and smallest element is : 1 +``` + +## Advantages and Disadvantages of Heaps: + +## Advantages: + +**Efficient:** Heap queues, implemented in Python's heapq module, offer remarkable efficiency in managing priority queues and heaps. With logarithmic time complexity for key operations, they are widely favored in various applications for their performance. + +**Space-efficient:** Leveraging an array-based representation, heap queues optimize memory usage compared to node-based structures like linked lists. This design minimizes overhead, enhancing efficiency in memory management. + +**Ease of Use:** Python's heap queues boast a user-friendly API, simplifying fundamental operations such as insertion, deletion, and retrieval. This simplicity contributes to rapid development and code maintenance. + +**Flexibility:** Beyond their primary use in priority queues and heaps, Python's heap queues lend themselves to diverse applications. They can be adapted to implement various data structures, including binary trees, showcasing their versatility and broad utility across different domains. + +## Disadvantages: + +**Limited functionality:** Heap queues are primarily designed for managing priority queues and heaps, and may not be suitable for more complex data structures and algorithms. + +**No random access:** Heap queues do not support random access to elements, making it difficult to access elements in the middle of the heap or modify elements that are not at the top of the heap. + +**No sorting:** Heap queues do not support sorting, so if you need to sort elements in a specific order, you will need to use a different data structure or algorithm. + +**Not thread-safe:** Heap queues are not thread-safe, meaning that they may not be suitable for use in multi-threaded applications where data synchronization is critical. + +## Real-Life Examples of Heaps: + +1. **Priority Queues:** +Heaps are commonly used to implement priority queues, which are used in various algorithms like Dijkstra's shortest path algorithm and Prim's minimum spanning tree algorithm. + +2. **Scheduling Algorithms:** +Heaps are used in job scheduling algorithms where tasks with the highest priority need to be processed first. + +3. **Merge K Sorted Lists:** +Heaps can be used to efficiently merge multiple sorted lists into a single sorted list. + +4. **Real-Time Event Simulation:** +Heaps are used in event-driven simulators to manage events scheduled to occur at future times. + +5. **Median Finding Algorithm:** +Heaps can be used to maintain a dynamic set of numbers to find the median efficiently. diff --git a/contrib/ds-algorithms/images/Dijkstra's_algorithm_photo1.png b/contrib/ds-algorithms/images/Dijkstra's_algorithm_photo1.png new file mode 100644 index 00000000..b937f046 Binary files /dev/null and b/contrib/ds-algorithms/images/Dijkstra's_algorithm_photo1.png differ diff --git a/contrib/ds-algorithms/images/Dijkstra's_algorithm_photo2.png b/contrib/ds-algorithms/images/Dijkstra's_algorithm_photo2.png new file mode 100644 index 00000000..e1cacef1 Binary files /dev/null and b/contrib/ds-algorithms/images/Dijkstra's_algorithm_photo2.png differ diff --git a/contrib/ds-algorithms/images/Dijkstra's_algorithm_photo3.png b/contrib/ds-algorithms/images/Dijkstra's_algorithm_photo3.png new file mode 100644 index 00000000..a5b69f9d Binary files /dev/null and b/contrib/ds-algorithms/images/Dijkstra's_algorithm_photo3.png differ diff --git a/contrib/ds-algorithms/images/Dijkstra's_algorithm_photo4.png b/contrib/ds-algorithms/images/Dijkstra's_algorithm_photo4.png new file mode 100644 index 00000000..54d1889c Binary files /dev/null and b/contrib/ds-algorithms/images/Dijkstra's_algorithm_photo4.png differ diff --git a/contrib/ds-algorithms/images/Dijkstra's_algorithm_photo5.png b/contrib/ds-algorithms/images/Dijkstra's_algorithm_photo5.png new file mode 100644 index 00000000..a3a6d508 Binary files /dev/null and b/contrib/ds-algorithms/images/Dijkstra's_algorithm_photo5.png differ diff --git a/contrib/ds-algorithms/images/Dijkstra's_algorithm_photo6.png b/contrib/ds-algorithms/images/Dijkstra's_algorithm_photo6.png new file mode 100644 index 00000000..db7d948c Binary files /dev/null and b/contrib/ds-algorithms/images/Dijkstra's_algorithm_photo6.png differ diff --git a/contrib/ds-algorithms/images/Dijkstra's_algorithm_photo7.png b/contrib/ds-algorithms/images/Dijkstra's_algorithm_photo7.png new file mode 100644 index 00000000..0b4eaf89 Binary files /dev/null and b/contrib/ds-algorithms/images/Dijkstra's_algorithm_photo7.png differ diff --git a/contrib/ds-algorithms/images/binarytree.png b/contrib/ds-algorithms/images/binarytree.png new file mode 100644 index 00000000..4137cdfc Binary files /dev/null and b/contrib/ds-algorithms/images/binarytree.png differ diff --git a/contrib/ds-algorithms/images/inorder-traversal.png b/contrib/ds-algorithms/images/inorder-traversal.png new file mode 100644 index 00000000..61b32c70 Binary files /dev/null and b/contrib/ds-algorithms/images/inorder-traversal.png differ diff --git a/contrib/ds-algorithms/images/postorder-traversal.png b/contrib/ds-algorithms/images/postorder-traversal.png new file mode 100644 index 00000000..69ac6590 Binary files /dev/null and b/contrib/ds-algorithms/images/postorder-traversal.png differ diff --git a/contrib/ds-algorithms/images/preorder-traversal.png b/contrib/ds-algorithms/images/preorder-traversal.png new file mode 100644 index 00000000..e85a70d8 Binary files /dev/null and b/contrib/ds-algorithms/images/preorder-traversal.png differ diff --git a/contrib/ds-algorithms/images/traversal.png b/contrib/ds-algorithms/images/traversal.png new file mode 100644 index 00000000..556f1775 Binary files /dev/null and b/contrib/ds-algorithms/images/traversal.png differ diff --git a/contrib/ds-algorithms/index.md b/contrib/ds-algorithms/index.md index 31cff39b..0c29ec75 100644 --- a/contrib/ds-algorithms/index.md +++ b/contrib/ds-algorithms/index.md @@ -10,3 +10,17 @@ - [Greedy Algorithms](greedy-algorithms.md) - [Dynamic Programming](dynamic-programming.md) - [Linked list](linked-list.md) +- [Stacks in Python](stacks.md) +- [Sliding Window Technique](sliding-window.md) +- [Trie](trie.md) +- [Two Pointer Technique](two-pointer-technique.md) +- [Hashing through Linear Probing](hashing-linear-probing.md) +- [Hashing through Chaining](hashing-chaining.md) +- [Heaps](heaps.md) +- [Hash Tables, Sets, Maps](hash-tables.md) +- [Binary Tree](binary-tree.md) +- [AVL Trees](avl-trees.md) +- [Splay Trees](splay-trees.md) +- [Dijkstra's Algorithm](dijkstra.md) +- [Deque](deque.md) +- [Tree Traversals](tree-traversal.md) diff --git a/contrib/ds-algorithms/linked-list.md b/contrib/ds-algorithms/linked-list.md index ddbc6d56..59631e7c 100644 --- a/contrib/ds-algorithms/linked-list.md +++ b/contrib/ds-algorithms/linked-list.md @@ -1,6 +1,6 @@ # Linked List Data Structure -Link list is a linear data Structure which can be defined as collection of objects called nodes that are randomly stored in the memory. +Linked list is a linear data Structure which can be defined as collection of objects called nodes that are randomly stored in the memory. A node contains two types of metadata i.e. data stored at that particular address and the pointer which contains the address of the next node in the memory. The last element in a linked list features a null pointer. @@ -36,10 +36,10 @@ The smallest Unit: Node Now, we will see the types of linked list. There are mainly four types of linked list, -1. Singly Link list -2. Doubly link list -3. Circular link list -4. Doubly circular link list +1. Singly linked list +2. Doubly linked list +3. Circular linked list +4. Doubly circular linked list ## 1. Singly linked list. @@ -160,6 +160,18 @@ check the list is empty otherwise shift the head to next node. temp.next = None # Remove the last node by setting the next pointer of the second-last node to None ``` +### Reversing the linked list +```python + def reverseList(self): + prev = None + temp = self.head + while(temp): + nextNode = temp.next #Store the next node + temp.next = prev # Reverse the pointer of current node + prev = temp # Move prev pointer one step forward + temp = nextNode # Move temp pointer one step forward. + self.head = prev # Update the head pointer to last node +``` ### Search in a linked list ```python @@ -174,6 +186,8 @@ check the list is empty otherwise shift the head to next node. return f"Value '{value}' not found in the list" ``` +Connect all the code. + ```python if __name__ == '__main__': llist = LinkedList() @@ -197,13 +211,17 @@ check the list is empty otherwise shift the head to next node. #delete at the end llist.deleteFromEnd() # 2 3 56 9 4 10 - # Print the list + # Print the original list + llist.printList() + llist.reverseList() #10 4 9 56 3 2 + # Print the reversed list llist.printList() ``` ## Output: -2 3 56 9 4 10 +2 3 56 9 4 10 +10 4 9 56 3 2 ## Real Life uses of Linked List diff --git a/contrib/ds-algorithms/Queues.md b/contrib/ds-algorithms/queues.md similarity index 100% rename from contrib/ds-algorithms/Queues.md rename to contrib/ds-algorithms/queues.md diff --git a/contrib/ds-algorithms/sliding-window.md b/contrib/ds-algorithms/sliding-window.md new file mode 100644 index 00000000..72aa1915 --- /dev/null +++ b/contrib/ds-algorithms/sliding-window.md @@ -0,0 +1,249 @@ +# Sliding Window Technique + +The sliding window technique is a fundamental approach used to solve problems involving arrays, lists, or sequences. It's particularly useful when you need to calculate something over a subarray or sublist of fixed size that slides over the entire array. + +In easy words, It is the transformation of the nested loops into the single loop +## Concept + +The sliding window technique involves creating a window (a subarray or sublist) that moves or "slides" across the entire array. This window can either be fixed in size or dynamically resized. By maintaining and updating this window as it moves, you can optimize certain computations, reducing time complexity. + +## Types of Sliding Windows + +1. **Fixed Size Window**: The window size remains constant as it slides from the start to the end of the array. +2. **Variable Size Window**: The window size can change based on certain conditions, such as the sum of elements within the window meeting a specified target. + +## Steps to Implement a Sliding Window + +1. **Initialize the Window**: Set the initial position of the window and any required variables (like sum, count, etc.). +2. **Expand the Window**: Add the next element to the window and update the relevant variables. +3. **Shrink the Window**: If needed, remove elements from the start of the window and update the variables. +4. **Slide the Window**: Move the window one position to the right by including the next element and possibly excluding the first element. +5. **Repeat**: Continue expanding, shrinking, and sliding the window until you reach the end of the array. + +## Example Problems + +### 1. Maximum Sum Subarray of Fixed Size K + +Given an array of integers and an integer k, find the maximum sum of a subarray of size k. + +**Steps:** + +1. Initialize the sum of the first k elements. +2. Slide the window from the start of the array to the end, updating the sum by subtracting the element that is left behind and adding the new element. +3. Track the maximum sum encountered. + +**Python Code:** + +```python +def max_sum_subarray(arr, k): + n = len(arr) + if n < k: + return None + + # Compute the sum of the first window + window_sum = sum(arr[:k]) + max_sum = window_sum + + # Slide the window from start to end + for i in range(n - k): + window_sum = window_sum - arr[i] + arr[i + k] + max_sum = max(max_sum, window_sum) + + return max_sum + +# Example usage: +arr = [1, 3, 2, 5, 1, 1, 6, 2, 8, 5] +k = 3 +print(max_sum_subarray(arr, k)) # Output: 16 +``` + +### 2. Longest Substring Without Repeating Characters + +Given a string, find the length of the longest substring without repeating characters. + +**Steps:** + +1. Use two pointers to represent the current window. +2. Use a set to track characters in the current window. +3. Expand the window by moving the right pointer. +4. If a duplicate character is found, shrink the window by moving the left pointer until the duplicate is removed. + +**Python Code:** + +```python +def longest_unique_substring(s): + n = len(s) + char_set = set() + left = 0 + max_length = 0 + + for right in range(n): + while s[right] in char_set: + char_set.remove(s[left]) + left += 1 + char_set.add(s[right]) + max_length = max(max_length, right - left + 1) + + return max_length + +# Example usage: +s = "abcabcbb" +print(longest_unique_substring(s)) # Output: 3 +``` +## 3. Minimum Size Subarray Sum + +Given an array of positive integers and a positive integer `s`, find the minimal length of a contiguous subarray of which the sum is at least `s`. If there isn't one, return 0 instead. + +### Steps: +1. Use two pointers, `left` and `right`, to define the current window. +2. Expand the window by moving `right` and adding `arr[right]` to `current_sum`. +3. If `current_sum` is greater than or equal to `s`, update `min_length` and shrink the window from the left by moving `left` and subtracting `arr[left]` from `current_sum`. +4. Repeat until `right` has traversed the array. + +### Python Code: +```python +def min_subarray_len(s, arr): + n = len(arr) + left = 0 + current_sum = 0 + min_length = float('inf') + + for right in range(n): + current_sum += arr[right] + + while current_sum >= s: + min_length = min(min_length, right - left + 1) + current_sum -= arr[left] + left += 1 + + return min_length if min_length != float('inf') else 0 + +# Example usage: +arr = [2, 3, 1, 2, 4, 3] +s = 7 +print(min_subarray_len(s, arr)) # Output: 2 (subarray [4, 3]) +``` + +## 4. Longest Substring with At Most K Distinct Characters + +Given a string `s` and an integer `k`, find the length of the longest substring that contains at most `k` distinct characters. + +### Steps: +1. Use two pointers, `left` and `right`, to define the current window. +2. Use a dictionary `char_count` to count characters in the window. +3. Expand the window by moving `right` and updating `char_count`. +4. If `char_count` has more than `k` distinct characters, shrink the window from the left by moving `left` and updating `char_count`. +5. Keep track of the maximum length of the window with at most `k` distinct characters. + +### Python Code: +```python +def longest_substring_k_distinct(s, k): + n = len(s) + char_count = {} + left = 0 + max_length = 0 + + for right in range(n): + char_count[s[right]] = char_count.get(s[right], 0) + 1 + + while len(char_count) > k: + char_count[s[left]] -= 1 + if char_count[s[left]] == 0: + del char_count[s[left]] + left += 1 + + max_length = max(max_length, right - left + 1) + + return max_length + +# Example usage: +s = "eceba" +k = 2 +print(longest_substring_k_distinct(s, k)) # Output: 3 (substring "ece") +``` + +## 5. Maximum Number of Vowels in a Substring of Given Length + +Given a string `s` and an integer `k`, return the maximum number of vowel letters in any substring of `s` with length `k`. + +### Steps: +1. Use a sliding window of size `k`. +2. Keep track of the number of vowels in the current window. +3. Expand the window by adding the next character and update the count if it's a vowel. +4. If the window size exceeds `k`, remove the leftmost character and update the count if it's a vowel. +5. Track the maximum number of vowels found in any window of size `k`. + +### Python Code: +```python +def max_vowels(s, k): + vowels = set('aeiou') + max_vowel_count = 0 + current_vowel_count = 0 + + for i in range(len(s)): + if s[i] in vowels: + current_vowel_count += 1 + if i >= k: + if s[i - k] in vowels: + current_vowel_count -= 1 + max_vowel_count = max(max_vowel_count, current_vowel_count) + + return max_vowel_count + +# Example usage: +s = "abciiidef" +k = 3 +print(max_vowels(s, k)) # Output: 3 (substring "iii") +``` + +## 6. Subarray Product Less Than K + +Given an array of positive integers `nums` and an integer `k`, return the number of contiguous subarrays where the product of all the elements in the subarray is less than `k`. + +### Steps: +1. Use two pointers, `left` and `right`, to define the current window. +2. Expand the window by moving `right` and multiplying `product` by `nums[right]`. +3. If `product` is greater than or equal to `k`, shrink the window from the left by moving `left` and dividing `product` by `nums[left]`. +4. For each position of `right`, the number of valid subarray ending at `right` is `right - left + 1`. +5. Sum these counts to get the total number of subarray with product less than `k`. + +### Python Code: +```python +def num_subarray_product_less_than_k(nums, k): + if k <= 1: + return 0 + + product = 1 + left = 0 + count = 0 + + for right in range(len(nums)): + product *= nums[right] + + while product >= k: + product /= nums[left] + left += 1 + + count += right - left + 1 + + return count + +# Example usage: +nums = [10, 5, 2, 6] +k = 100 +print(num_subarray_product_less_than_k(nums, k)) # Output: 8 +``` + +## Advantages + +- **Efficiency**: Reduces the time complexity from O(n^2) to O(n) for many problems. +- **Simplicity**: Provides a straightforward way to manage subarrays/substrings with overlapping elements. + +## Applications + +- Finding the maximum or minimum sum of subarrays of fixed size. +- Detecting unique elements in a sequence. +- Solving problems related to dynamic programming with fixed constraints. +- Efficiently managing and processing streaming data or real-time analytics. + +By using the sliding window technique, you can tackle a wide range of problems in a more efficient manner. diff --git a/contrib/ds-algorithms/sorting-algorithms.md b/contrib/ds-algorithms/sorting-algorithms.md index 2edc265c..a3cd72e2 100644 --- a/contrib/ds-algorithms/sorting-algorithms.md +++ b/contrib/ds-algorithms/sorting-algorithms.md @@ -465,3 +465,154 @@ print("Sorted array:", arr) # Output: [1, 2, 3, 5, 8] - **Worst Case:** `𝑂(𝑛log𝑛)`. Building the heap takes `𝑂(𝑛)` time, and each of the 𝑛 element extractions takes `𝑂(log𝑛)` time. - **Best Case:** `𝑂(𝑛log𝑛)`. Even if the array is already sorted, heap sort will still build the heap and perform the extractions. - **Average Case:** `𝑂(𝑛log𝑛)`. Similar to the worst-case, the overall complexity remains `𝑂(𝑛log𝑛)` because each insertion and deletion in a heap takes `𝑂(log𝑛)` time, and these operations are performed 𝑛 times. + + ## 7. Radix Sort +Radix Sort is a non-comparative integer sorting algorithm that sorts numbers by processing individual digits. It processes digits from the least significant digit (LSD) to the most significant digit (MSD) or vice versa. This algorithm is efficient for sorting numbers with a fixed number of digits. + +**Algorithm Overview:** +- **Digit by Digit sorting:** Radix sort processes the digits of the numbers starting from either the least significant digit (LSD) or the most significant digit (MSD). Typically, LSD is used. +- **Stable Sort:** A stable sorting algorithm like Counting Sort or Bucket Sort is used as an intermediate sorting technique. Radix Sort relies on this stability to maintain the relative order of numbers with the same digit value. +- **Multiple passes:** The algorithm performs multiple passes over the numbers, one for each digit, from the least significant to the most significant. + +### Radix Sort Code in Python + +```python +def counting_sort(arr, exp): + n = len(arr) + output = [0] * n + count = [0] * 10 + + for i in range(n): + index = arr[i] // exp + count[index % 10] += 1 + + for i in range(1, 10): + count[i] += count[i - 1] + + i = n - 1 + while i >= 0: + index = arr[i] // exp + output[count[index % 10] - 1] = arr[i] + count[index % 10] -= 1 + i -= 1 + + for i in range(n): + arr[i] = output[i] + +def radix_sort(arr): + max_num = max(arr) + exp = 1 + while max_num // exp > 0: + counting_sort(arr, exp) + exp *= 10 + +# Example usage +arr = [170, 45, 75, 90] +print("Original array:", arr) +radix_sort(arr) +print("Sorted array:", arr) +``` + +### Complexity Analysis + - **Time Complexity:** O(d * (n + k)) for all cases. Radix Sort always processes each digit of every number in the array. + - **Space Complexity:** O(n + k). This is due to the space required for: +- The output array used in Counting Sort, which is of size n. +- The count array used in Counting Sort, which is of size k. + +## 8. Counting Sort +Counting sort is a sorting technique based on keys between a specific range. It works by counting the number of objects having distinct key values (kind of hashing). Then do some arithmetic to calculate the position of each object in the output sequence. + +**Algorithm Overview:** +- Convert the input string into a list of characters. +- Count the occurrence of each character in the list using the collections.Counter() method. +- Sort the keys of the resulting Counter object to get the unique characters in the list in sorted order. +- For each character in the sorted list of keys, create a list of repeated characters using the corresponding count from the Counter object. +- Concatenate the lists of repeated characters to form the sorted output list. + + +### Counting Sort Code in Python using counter method. + +```python +from collections import Counter + +def counting_sort(arr): + count = Counter(arr) + output = [] + for c in sorted(count.keys()): + output += * count + return output + +arr = "geeksforgeeks" +arr = list(arr) +arr = counting_sort(arr) +output = ''.join(arr) +print("Sorted character array is", output) + +``` +### Counting Sort Code in Python using sorted() and reduce(): + +```python +from functools import reduce +string = "geeksforgeeks" +sorted_str = reduce(lambda x, y: x+y, sorted(string)) +print("Sorted string:", sorted_str) +``` + +### Complexity Analysis + - **Time Complexity:** O(n+k) for all cases.No matter how the elements are placed in the array, the algorithm goes through n+k times + - **Space Complexity:** O(max). Larger the range of elements, larger is the space complexity. + + +## 9. Cyclic Sort + +### Theory +Cyclic Sort is an in-place sorting algorithm that is useful for sorting arrays where the elements are in a known range (e.g., 1 to N). The key idea behind the algorithm is that each number should be placed at its correct index. If we find a number that is not at its correct index, we swap it with the number at its correct index. This process is repeated until every number is at its correct index. + +### Algorithm +- Iterate over the array from the start to the end. +- For each element, check if it is at its correct index. +- If it is not at its correct index, swap it with the element at its correct index. +- Continue this process until the element at the current index is in its correct position. Move to the next index and repeat the process until the end of the array is reached. + +### Steps +- Start with the first element. +- Check if it is at the correct index (i.e., if arr[i] == i + 1). +- If it is not, swap it with the element at the index arr[i] - 1. +- Repeat step 2 for the current element until it is at the correct index. +- Move to the next element and repeat the process. + +### Code + +```python +def cyclic_sort(nums): + i = 0 + while i < len(nums): + correct_index = nums[i] - 1 + if nums[i] != nums[correct_index]: + nums[i], nums[correct_index] = nums[correct_index], nums[i] # Swap + else: + i += 1 + return nums +``` + +### Example +``` +arr = [3, 1, 5, 4, 2] +sorted_arr = cyclic_sort(arr) +print(sorted_arr) + ``` +### Output +``` +[1, 2, 3, 4, 5] +``` + +### Complexity Analysis +**Time Complexity:** + +The time complexity of Cyclic Sort is **O(n)**. +This is because in each cycle, each element is either placed in its correct position or a swap is made. Since each element is swapped at most once, the total number of swaps (and hence the total number of operations) is linear in the number of elements. + +**Space Complexity:** + +The space complexity of Cyclic Sort is **O(1)**. +This is because the algorithm only requires a constant amount of additional space beyond the input array. \ No newline at end of file diff --git a/contrib/ds-algorithms/splay-trees.md b/contrib/ds-algorithms/splay-trees.md new file mode 100644 index 00000000..ee900ed8 --- /dev/null +++ b/contrib/ds-algorithms/splay-trees.md @@ -0,0 +1,162 @@ +# Splay Tree + +In Data Structures and Algorithms, a **Splay Tree** is a self-adjusting binary search tree with the additional property that recently accessed elements are quick to access again. It performs basic operations such as insertion, search, and deletion in O(log n) amortized time. This is achieved by a process called **splaying**, where the accessed node is moved to the root through a series of tree rotations. + +## Points to be Remembered + +- **Splaying**: Moving the accessed node to the root using rotations. +- **Rotations**: Tree rotations (left and right) are used to balance the tree during splaying. +- **Self-adjusting**: The tree adjusts itself with each access, keeping frequently accessed nodes near the root. + +## Real Life Examples of Splay Trees + +- **Cache Implementation**: Frequently accessed data is kept near the top of the tree, making repeated accesses faster. +- **Networking**: Routing tables in network switches can use splay trees to prioritize frequently accessed routes. + +## Applications of Splay Trees + +Splay trees are used in various applications in Computer Science: + +- **Cache Implementations** +- **Garbage Collection Algorithms** +- **Data Compression Algorithms (e.g., LZ78)** + +Understanding these applications is essential for Software Development. + +## Operations in Splay Tree + +Key operations include: + +- **INSERT**: Insert a new element into the splay tree. +- **SEARCH**: Find the position of an element in the splay tree. +- **DELETE**: Remove an element from the splay tree. + +## Implementing Splay Tree in Python + +```python +class SplayTreeNode: + def __init__(self, key): + self.key = key + self.left = None + self.right = None + +class SplayTree: + def __init__(self): + self.root = None + + def insert(self, key): + self.root = self.splay_insert(self.root, key) + + def search(self, key): + self.root = self.splay_search(self.root, key) + return self.root + + def splay(self, root, key): + if not root or root.key == key: + return root + + if root.key > key: + if not root.left: + return root + if root.left.key > key: + root.left.left = self.splay(root.left.left, key) + root = self.rotateRight(root) + elif root.left.key < key: + root.left.right = self.splay(root.left.right, key) + if root.left.right: + root.left = self.rotateLeft(root.left) + return root if not root.left else self.rotateRight(root) + + else: + if not root.right: + return root + if root.right.key > key: + root.right.left = self.splay(root.right.left, key) + if root.right.left: + root.right = self.rotateRight(root.right) + elif root.right.key < key: + root.right.right = self.splay(root.right.right, key) + root = self.rotateLeft(root) + return root if not root.right else self.rotateLeft(root) + + def splay_insert(self, root, key): + if not root: + return SplayTreeNode(key) + + root = self.splay(root, key) + + if root.key == key: + return root + + new_node = SplayTreeNode(key) + + if root.key > key: + new_node.right = root + new_node.left = root.left + root.left = None + else: + new_node.left = root + new_node.right = root.right + root.right = None + + return new_node + + def splay_search(self, root, key): + return self.splay(root, key) + + def rotateRight(self, node): + temp = node.left + node.left = temp.right + temp.right = node + return temp + + def rotateLeft(self, node): + temp = node.right + node.right = temp.left + temp.left = node + return temp + + def preOrder(self, root): + if root: + print(root.key, end=' ') + self.preOrder(root.left) + self.preOrder(root.right) + +#Example usage: +splay_tree = SplayTree() +splay_tree.insert(50) +splay_tree.insert(30) +splay_tree.insert(20) +splay_tree.insert(40) +splay_tree.insert(70) +splay_tree.insert(60) +splay_tree.insert(80) + +print("Preorder traversal of the Splay tree is:") +splay_tree.preOrder(splay_tree.root) + +splay_tree.search(60) + +print("\nSplay tree after search operation for key 60:") +splay_tree.preOrder(splay_tree.root) +``` + +## Output + +```markdown +Preorder traversal of the Splay tree is: +50 30 20 40 70 60 80 + +Splay tree after search operation for key 60: +60 50 30 20 40 70 80 +``` + +## Complexity Analysis + +The worst-case time complexities of the main operations in a Splay Tree are as follows: + +- **Insertion**: (O(n)). In the worst case, insertion may take linear time if the tree is highly unbalanced. +- **Search**: (O(n)). In the worst case, searching for a node may take linear time if the tree is highly unbalanced. +- **Deletion**: (O(n)). In the worst case, deleting a node may take linear time if the tree is highly unbalanced. + +While these operations can take linear time in the worst case, the splay operation ensures that the tree remains balanced over a sequence of operations, leading to better average-case performance. \ No newline at end of file diff --git a/contrib/ds-algorithms/stacks.md b/contrib/ds-algorithms/stacks.md new file mode 100644 index 00000000..428a1938 --- /dev/null +++ b/contrib/ds-algorithms/stacks.md @@ -0,0 +1,116 @@ +# Stacks in Python + +In Data Structures and Algorithms, a stack is a linear data structure that complies with the Last In, First Out (LIFO) rule. It works by use of two fundamental techniques: **PUSH** which inserts an element on top of the stack and **POP** which takes out the topmost element.This concept is similar to a stack of plates in a cafeteria. Stacks are usually used for handling function calls, expression evaluation, and parsing in programming. Indeed, they are efficient in managing memory as well as tracking program state. + +## Points to be Remebered + +- A stack is a collection of data items that can be accessed at only one end, called **TOP**. +- Items can be inserted and deleted in a stack only at the TOP. +- The last item inserted in a stack is the first one to be deleted. +- Therefore, a stack is called a **Last-In-First-Out (LIFO)** data structure. + +## Real Life Examples of Stacks + +- **PILE OF BOOKS** - Suppose a set of books are placed one over the other in a pile. When you remove books from the pile, the topmost book will be removed first. Similarly, when you have to add a book to the pile, the book will be placed at the top of the file. + +- **PILE OF PLATES** - The first plate begins the pile. The second plate is placed on the top of the first plate and the third plate is placed on the top of the second plate, and so on. In general, if you want to add a plate to the pile, you can keep it on the top of the pile. Similarly, if you want to remove a plate, you can remove the plate from the top of the pile. + +- **BANGLES IN A HAND** - When a person wears bangles, the last bangle worn is the first one to be removed. + +## Applications of Stacks + +Stacks are widely used in Computer Science: + +- Function call management +- Maintaining the UNDO list for the application +- Web browser *history management* +- Evaluating expressions +- Checking the nesting of parentheses in an expression +- Backtracking algorithms (Recursion) + +Understanding these applications is essential for Software Development. + +## Operations on a Stack + +Key operations on a stack include: + +- **PUSH** - It is the process of inserting a new element on the top of a stack. +- **OVERFLOW** - A situation when we are pushing an item in a stack that is full. +- **POP** - It is the process of deleting an element from the top of a stack. +- **UNDERFLOW** - A situation when we are popping item from an empty stack. +- **PEEK** - It is the process of getting the most recent value of stack *(i.e. the value at the top of the stack)* +- **isEMPTY** - It is the function which return true if stack is empty else false. +- **SHOW** -Displaying stack items. + +## Implementing Stacks in Python + +```python +def isEmpty(S): + if len(S) == 0: + return True + else: + return False + +def Push(S, item): + S.append(item) + +def Pop(S): + if isEmpty(S): + return "Underflow" + else: + val = S.pop() + return val + +def Peek(S): + if isEmpty(S): + return "Underflow" + else: + top = len(S) - 1 + return S[top] + +def Show(S): + if isEmpty(S): + print("Sorry, No items in Stack") + else: + print("(Top)", end=' ') + t = len(S) - 1 + while t >= 0: + print(S[t], "<", end=' ') + t -= 1 + print() + +stack = [] # initially stack is empty + +Push(stack, 5) +Push(stack, 10) +Push(stack, 15) + +print("Stack after Push operations:") +Show(stack) +print("Peek operation:", Peek(stack)) +print("Pop operation:", Pop(stack)) +print("Stack after Pop operation:") +Show(stack) +``` + +## Output + +```markdown +Stack after Push operations: + +(Top) 15 < 10 < 5 < + +Peek operation: 15 + +Pop operation: 15 + +Stack after Pop operation: + +(Top) 10 < 5 < +``` + +## Complexity Analysis + +- **Worst case**: `O(n)` This occurs when the stack is full, it is dominated by the usage of Show operation. +- **Best case**: `O(1)` When the operations like isEmpty, Push, Pop and Peek are used, they have a constant time complexity of O(1). +- **Average case**: `O(n)` The average complexity is likely to be lower than O(n), as the stack is not always full. diff --git a/contrib/ds-algorithms/tree-traversal.md b/contrib/ds-algorithms/tree-traversal.md new file mode 100644 index 00000000..4ec72ee8 --- /dev/null +++ b/contrib/ds-algorithms/tree-traversal.md @@ -0,0 +1,195 @@ +# Tree Traversal Algorithms + +Tree Traversal refers to the process of visiting or accessing each node of the tree exactly once in a certain order. Tree traversal algorithms help us to visit and process all the nodes of the tree. Since tree is not a linear data structure, there are multiple nodes which we can visit after visiting a certain node. There are multiple tree traversal techniques which decide the order in which the nodes of the tree are to be visited. + + +A Tree Data Structure can be traversed in following ways: + - **Level Order Traversal or Breadth First Search or BFS** + - **Depth First Search or DFS** + - Inorder Traversal + - Preorder Traversal + - Postorder Traversal + +  + + + +## Binary Tree Structure + Before diving into traversal techniques, let's define a simple binary tree node structure: + +) + + ```python +class Node: + def __init__(self, key): + self.leftChild = None + self.rightChild = None + self.data = key + +# Main class +if __name__ == "__main__": + root = Node(1) + root.leftChild = Node(2) + root.rightChild = Node(3) + root.leftChild.leftChild = Node(4) + root.leftChild.rightChild = Node(5) + root.rightChild.leftChild = Node(6) + root.rightChild.rightChild = Node(6) +``` + +## Level Order Traversal +When the nodes of the tree are wrapped in a level-wise mode from left to right, then it represents the level order traversal. We can use a queue data structure to execute a level order traversal. + +### Algorithm + - Create an empty queue Q + - Enqueue the root node of the tree to Q + - Loop while Q is not empty + - Dequeue a node from Q and visit it + - Enqueue the left child of the dequeued node if it exists + - Enqueue the right child of the dequeued node if it exists + +### code for level order traversal in python +```python +def printLevelOrder(root): + if root is None: + return + + # Create an empty queue + queue = [] + + # Enqueue Root and initialize height + queue.append(root) + + while(len(queue) > 0): + + # Print front of queue and + # remove it from queue + print(queue[0].data, end=" ") + node = queue.pop(0) + + # Enqueue left child + if node.left is not None: + queue.append(node.left) + + # Enqueue right child + if node.right is not None: + queue.append(node.right) +``` + +**output** + +` Inorder traversal of binary tree is : +1 2 3 4 5 6 7 ` + + + +## Depth First Search +When we do a depth-first traversal, we travel in one direction up to the bottom first, then turn around and go the other way. There are three kinds of depth-first traversals. + +## 1. Inorder Traversal + +In this traversal method, the left subtree is visited first, then the root and later the right sub-tree. We should always remember that every node may represent a subtree itself. + +`Note :` If a binary search tree is traversed in-order, the output will produce sorted key values in an ascending order. + + + +**The order:** Left -> Root -> Right + +### Algorithm + - Traverse the left subtree. + - Visit the root node. + - Traverse the right subtree. + +### code for inorder traversal in python +```python +def printInorder(root): + if root: + # First recur on left child + printInorder(root.left) + + # Then print the data of node + print(root.val, end=" "), + + # Now recur on right child + printInorder(root.right) +``` + +**output** + +` Inorder traversal of binary tree is : +4 2 5 1 6 3 7 ` + + +## 2. Preorder Traversal + +In this traversal method, the root node is visited first, then the left subtree and finally the right subtree. + +) + +**The order:** Root -> Left -> Right + +### Algorithm + - Visit the root node. + - Traverse the left subtree. + - Traverse the right subtree. + +### code for preorder traversal in python +```python +def printPreorder(root): + if root: + # First print the data of node + print(root.val, end=" "), + + # Then recur on left child + printPreorder(root.left) + + # Finally recur on right child + printPreorder(root.right) +``` + +**output** + +` Inorder traversal of binary tree is : +1 2 4 5 3 6 7 ` + +## 3. Postorder Traversal + +In this traversal method, the root node is visited last, hence the name. First we traverse the left subtree, then the right subtree and finally the root node. + + + +**The order:** Left -> Right -> Root + +### Algorithm + - Traverse the left subtree. + - Traverse the right subtree. + - Visit the root node. + +### code for postorder traversal in python +```python +def printPostorder(root): + if root: + # First recur on left child + printPostorder(root.left) + + # The recur on right child + printPostorder(root.right) + + # Now print the data of node + print(root.val, end=" ") +``` + +**output** + +` Inorder traversal of binary tree is : +4 5 2 6 7 3 1 ` + + +## Complexity Analysis + - **Time Complexity:** All three tree traversal methods (Inorder, Preorder, and Postorder) have a time complexity of `𝑂(𝑛)`, where 𝑛 is the number of nodes in the tree. + - **Space Complexity:** The space complexity is influenced by the recursion stack. In the worst case, the depth of the recursion stack can go up to `𝑂(ℎ)`, where ℎ is the height of the tree. + + + + diff --git a/contrib/ds-algorithms/trie.md b/contrib/ds-algorithms/trie.md new file mode 100644 index 00000000..0ccfbaad --- /dev/null +++ b/contrib/ds-algorithms/trie.md @@ -0,0 +1,152 @@ +# Trie + +A Trie is a tree-like data structure used for storing a dynamic set of strings where the keys are usually strings. It is also known as prefix tree or digital tree. + +>Trie is a type of search tree, where each node represents a single character of a string. + +>Nodes are linked in such a way that they form a tree, where each path from the root to a leaf node represents a unique string stored in the Trie. + +## Characteristics of Trie +- **Prefix Matching**: Tries are particularly useful for prefix matching operations. Any node in the Trie represents a common prefix of all strings below it. +- **Space Efficiency**: Tries can be more space-efficient than other data structures like hash tables for storing large sets of strings with common prefixes. +- **Time Complexity**: Insertion, deletion, and search operations in a Trie have a time complexity of +𝑂(𝑚), where m is the length of the string. This makes Tries very efficient for these operations. + +## Structure of Trie + +Trie mainly consists of three parts: +- **Root**: The root of a Trie is an empty node that does not contain any character. +- **Edges**: Each edge in the Trie represents a character in the alphabet of the stored strings. +- **Nodes**: Each node contains a character and possibly additional information, such as a boolean flag indicating if the node represents the end of a valid string. + +To implement the nodes of trie. We use Classes in Python. Each node is an object of the Node Class. + +Node Class have mainly two components +- *Array of size 26*: It is used to represent the 26 alphabets. Initially all are None. While inserting the words, then array will be filled with object of child nodes. +- *End of word*: It is used to represent the end of word while inserting. + +Code Block of Node Class : + +```python +class Node: + def __init__(self): + self.alphabets = [None] * 26 + self.end_of_word = 0 +``` + +Now we need to implement Trie. We create another class named Trie with some methods like Insertion, Searching and Deletion. + +**Initialization:** In this, we initializes the Trie with a `root` node. + +Code Implementation of Initialization: + +```python +class Trie: + def __init__(self): + self.root = Node() +``` + +## Operations on Trie + +1. **Insertion**: Inserts the word into the Trie. This method takes `word` as parameter. For each character in the word, it checks if there is a corresponding child node. If not, it creates a new `Node`. After processing all the characters in word, it increments the `end_of_word` value of the last node. + +Code Implementation of Insertion: +```python +def insert(self, word): + node = self.root + for char in word: + index = ord(char) - ord('a') + if not node.alphabets[index]: + node.alphabets[index] = Node() + node = node.alphabets[index] + node.end_of_word += 1 +``` + +2. **Searching**: Search the `word` in trie. Searching process starts from the `root` node. Each character of the `word` is processed. After traversing the whole word in trie, it return the count of words. + +There are two cases in Searching: +- *Word Not found*: It happens when the word we search not present in the trie. This case will occur, if the value of `alphabets` array at that character is `None` or if the value of `end_of_word` of the node, reached after traversing the whole word is `0`. +- *Word found*: It happens when the search word is present in the Trie. This case will occur, when the `end_of_word` value is greater than `0` of the node after traversing the whole word. + +Code Implementation of Searching: +```python + def Search(self, word): + node = self.root + for char in word: + index = ord(char) - ord('a') + if not node.alphabets[index]: + return 0 + node = node.alphabets[index] + return node.end_of_word +``` + +3. **Deletion**: To delete a string, follow the path of the string. If the end node is reached and `end_of_word` is greater than `0` then decrement the value. + +Code Implementation of Deletion: + +```python +def delete(self, word): + node = self.root + for char in word: + index = ord(char) - ord('a') + node = node.alphabets[index] + if node.end_of_word: + node.end_of_word-=1 +``` + +Python Code to implement Trie: + +```python +class Node: + def __init__(self): + self.alphabets = [None] * 26 + self.end_of_word = 0 + +class Trie: + def __init__(self): + self.root = Node() + + def insert(self, word): + node = self.root + for char in word: + index = ord(char) - ord('a') + if not node.alphabets[index]: + node.alphabets[index] = Node() + node = node.alphabets[index] + node.end_of_word += 1 + + def Search(self, word): + node = self.root + for char in word: + index = ord(char) - ord('a') + if not node.alphabets[index]: + return 0 + node = node.alphabets[index] + return node.end_of_word + + def delete(self, word): + node = self.root + for char in word: + index = ord(char) - ord('a') + node = node.alphabets[index] + if node.end_of_word: + node.end_of_word-=1 + +if __name__ == "__main__": + trie = Trie() + + word1 = "apple" + word2 = "app" + word3 = "bat" + + trie.insert(word1) + trie.insert(word2) + trie.insert(word3) + + print(trie.Search(word1)) + print(trie.Search(word2)) + print(trie.Search(word3)) + + trie.delete(word2) + print(trie.Search(word2)) +``` diff --git a/contrib/ds-algorithms/two-pointer-technique.md b/contrib/ds-algorithms/two-pointer-technique.md new file mode 100644 index 00000000..6b8720a5 --- /dev/null +++ b/contrib/ds-algorithms/two-pointer-technique.md @@ -0,0 +1,132 @@ +# Two-Pointer Technique + +--- + +- The two-pointer technique is a popular algorithmic strategy used to solve various problems efficiently. This technique involves using two pointers (or indices) to traverse through data structures such as arrays or linked lists. +- The pointers can move in different directions, allowing for efficient processing of elements to achieve the desired results. + +## Common Use Cases + +1. **Finding pairs in a sorted array that sum to a target**: One pointer starts at the beginning and the other at the end. +2. **Reversing a linked list**: One pointer starts at the head, and the other at the next node, progressing through the list. +3. **Removing duplicates from a sorted array**: One pointer keeps track of the unique elements, and the other traverses the array. +4. **Merging two sorted arrays**: Two pointers are used to iterate through the arrays and merge them. + +## Example 1: Finding Pairs with a Given Sum + +### Problem Statement + +Given a sorted array of integers and a target sum, find all pairs in the array that sum up to the target. + +### Approach + +1. Initialize two pointers: one at the beginning (`left`) and one at the end (`right`) of the array. +2. Calculate the sum of the elements at the `left` and `right` pointers. +3. If the sum is equal to the target, record the pair and move both pointers inward. +4. If the sum is less than the target, move the `left` pointer to the right to increase the sum. +5. If the sum is greater than the target, move the `right` pointer to the left to decrease the sum. +6. Repeat the process until the `left` pointer is not less than the `right` pointer. + +### Example Code + +```python +def find_pairs_with_sum(arr, target): + left = 0 + right = len(arr) - 1 + pairs = [] + + while left < right: + current_sum = arr[left] + arr[right] + + if current_sum == target: + pairs.append((arr[left], arr[right])) + left += 1 + right -= 1 + elif current_sum < target: + left += 1 + else: + right -= 1 + + return pairs + +# Example usage +arr = [1, 2, 3, 4, 5, 6, 7, 8, 9] +target = 10 +result = find_pairs_with_sum(arr, target) +print("Pairs with sum", target, "are:", result) + ``` + +## Example 2: Removing Duplicates from a Sorted Array + +### Problem Statement +Given a sorted array, remove the duplicates in place such that each element appears only once and return the new length of the array. + +### Approach +1. If the array is empty, return 0. +2. Initialize a slow pointer at the beginning of the array. +3. Use a fast pointer to traverse through the array. +4. Whenever the element at the fast pointer is different from the element at the slow pointer, increment the slow pointer and update the element at the slow pointer with the element at the fast pointer. +5. Continue this process until the fast pointer reaches the end of the array. +6. The slow pointer will indicate the position of the last unique element. + +### Example Code + +```python +def remove_duplicates(arr): + if not arr: + return 0 + + slow = 0 + + for fast in range(1, len(arr)): + if arr[fast] != arr[slow]: + slow += 1 + arr[slow] = arr[fast] + + return slow + 1 + +# Example usage +arr = [1, 1, 2, 2, 3, 4, 4, 5] +new_length = remove_duplicates(arr) +print("Array after removing duplicates:", arr[:new_length]) +print("New length of array:", new_length) +``` +# Advantages of the Two-Pointer Technique + +Here are some key benefits of using the two-pointer technique: + +## 1. **Improved Time Complexity** + +It often reduces the time complexity from O(n^2) to O(n), making it significantly faster for many problems. + +### Example +- **Finding pairs with a given sum**: Efficiently finds pairs in O(n) time. + +## 2. **Simplicity** + +The implementation is straightforward, using basic operations like incrementing or decrementing pointers. + +### Example +- **Removing duplicates from a sorted array**: Easy to implement and understand. + +## 3. **In-Place Solutions** + +Many problems can be solved in place, requiring no extra space beyond the input data. + +### Example +- **Reversing a linked list**: Adjusts pointers within the existing nodes. + +## 4. **Versatility** + +Applicable to a wide range of problems, from arrays and strings to linked lists. + +### Example +- **Merging two sorted arrays**: Efficiently merges using two pointers. + +## 5. **Efficiency** + +Minimizes redundant operations and enhances performance, especially with large data sets. + +### Example +- **Partitioning problems**: Efficiently partitions elements with minimal operations. + diff --git a/contrib/machine-learning/ArtificialNeuralNetwork.md b/contrib/machine-learning/ann.md similarity index 100% rename from contrib/machine-learning/ArtificialNeuralNetwork.md rename to contrib/machine-learning/ann.md diff --git a/contrib/machine-learning/assets/XG_1.webp b/contrib/machine-learning/assets/XG_1.webp new file mode 100644 index 00000000..c693d3da Binary files /dev/null and b/contrib/machine-learning/assets/XG_1.webp differ diff --git a/contrib/machine-learning/assets/cnn-dropout.png b/contrib/machine-learning/assets/cnn-dropout.png new file mode 100644 index 00000000..9cb18f95 Binary files /dev/null and b/contrib/machine-learning/assets/cnn-dropout.png differ diff --git a/contrib/machine-learning/assets/cnn-filters.png b/contrib/machine-learning/assets/cnn-filters.png new file mode 100644 index 00000000..463ca600 Binary files /dev/null and b/contrib/machine-learning/assets/cnn-filters.png differ diff --git a/contrib/machine-learning/assets/cnn-flattened.png b/contrib/machine-learning/assets/cnn-flattened.png new file mode 100644 index 00000000..2d1ca6f2 Binary files /dev/null and b/contrib/machine-learning/assets/cnn-flattened.png differ diff --git a/contrib/machine-learning/assets/cnn-input_shape.png b/contrib/machine-learning/assets/cnn-input_shape.png new file mode 100644 index 00000000..34379f1d Binary files /dev/null and b/contrib/machine-learning/assets/cnn-input_shape.png differ diff --git a/contrib/machine-learning/assets/cnn-ouputs.png b/contrib/machine-learning/assets/cnn-ouputs.png new file mode 100644 index 00000000..27972265 Binary files /dev/null and b/contrib/machine-learning/assets/cnn-ouputs.png differ diff --git a/contrib/machine-learning/assets/cnn-padding.png b/contrib/machine-learning/assets/cnn-padding.png new file mode 100644 index 00000000..a441b2b9 Binary files /dev/null and b/contrib/machine-learning/assets/cnn-padding.png differ diff --git a/contrib/machine-learning/assets/cnn-pooling.png b/contrib/machine-learning/assets/cnn-pooling.png new file mode 100644 index 00000000..c3ada5cf Binary files /dev/null and b/contrib/machine-learning/assets/cnn-pooling.png differ diff --git a/contrib/machine-learning/assets/cnn-strides.png b/contrib/machine-learning/assets/cnn-strides.png new file mode 100644 index 00000000..26339a9f Binary files /dev/null and b/contrib/machine-learning/assets/cnn-strides.png differ diff --git a/contrib/machine-learning/assets/eda/bi-variate-analysis.png b/contrib/machine-learning/assets/eda/bi-variate-analysis.png new file mode 100644 index 00000000..076cc505 Binary files /dev/null and b/contrib/machine-learning/assets/eda/bi-variate-analysis.png differ diff --git a/contrib/machine-learning/assets/eda/correlation-analysis.png b/contrib/machine-learning/assets/eda/correlation-analysis.png new file mode 100644 index 00000000..e6f3ee60 Binary files /dev/null and b/contrib/machine-learning/assets/eda/correlation-analysis.png differ diff --git a/contrib/machine-learning/assets/eda/multi-variate-analysis.png b/contrib/machine-learning/assets/eda/multi-variate-analysis.png new file mode 100644 index 00000000..5dc042b9 Binary files /dev/null and b/contrib/machine-learning/assets/eda/multi-variate-analysis.png differ diff --git a/contrib/machine-learning/assets/eda/uni-variate-analysis1.png b/contrib/machine-learning/assets/eda/uni-variate-analysis1.png new file mode 100644 index 00000000..b4905dcf Binary files /dev/null and b/contrib/machine-learning/assets/eda/uni-variate-analysis1.png differ diff --git a/contrib/machine-learning/assets/eda/uni-variate-analysis2.png b/contrib/machine-learning/assets/eda/uni-variate-analysis2.png new file mode 100644 index 00000000..cf56c70a Binary files /dev/null and b/contrib/machine-learning/assets/eda/uni-variate-analysis2.png differ diff --git a/contrib/machine-learning/assets/km_.png b/contrib/machine-learning/assets/km_.png new file mode 100644 index 00000000..3f674126 Binary files /dev/null and b/contrib/machine-learning/assets/km_.png differ diff --git a/contrib/machine-learning/assets/km_2.png b/contrib/machine-learning/assets/km_2.png new file mode 100644 index 00000000..cf786cf2 Binary files /dev/null and b/contrib/machine-learning/assets/km_2.png differ diff --git a/contrib/machine-learning/assets/km_3.png b/contrib/machine-learning/assets/km_3.png new file mode 100644 index 00000000..ecd34ff5 Binary files /dev/null and b/contrib/machine-learning/assets/km_3.png differ diff --git a/contrib/machine-learning/assets/knm.png b/contrib/machine-learning/assets/knm.png new file mode 100644 index 00000000..4b7a2190 Binary files /dev/null and b/contrib/machine-learning/assets/knm.png differ diff --git a/contrib/machine-learning/assets/transformer-architecture.png b/contrib/machine-learning/assets/transformer-architecture.png new file mode 100644 index 00000000..2854ab0a Binary files /dev/null and b/contrib/machine-learning/assets/transformer-architecture.png differ diff --git a/contrib/machine-learning/binomial_distribution.md b/contrib/machine-learning/binomial-distribution.md similarity index 100% rename from contrib/machine-learning/binomial_distribution.md rename to contrib/machine-learning/binomial-distribution.md diff --git a/contrib/machine-learning/clustering.md b/contrib/machine-learning/clustering.md new file mode 100644 index 00000000..bc02d374 --- /dev/null +++ b/contrib/machine-learning/clustering.md @@ -0,0 +1,96 @@ +# Clustering + +Clustering is an unsupervised machine learning technique that groups a set of objects in such a way that objects in the same group (called a cluster) are more similar to each other than to those in other groups (clusters). This README provides an overview of clustering, including its fundamental concepts, types, algorithms, and how to implement it using Python. + +## Introduction + +Clustering is a technique used to find inherent groupings within data without pre-labeled targets. It is widely used in exploratory data analysis, pattern recognition, image analysis, information retrieval, and bioinformatics. + +## Concepts + +### Centroid + +A centroid is the center of a cluster. In the k-means clustering algorithm, for example, each cluster is represented by its centroid, which is the mean of all the data points in the cluster. + +### Distance Measure + +Distance measures are used to quantify the similarity or dissimilarity between data points. Common distance measures include Euclidean distance, Manhattan distance, and cosine similarity. + +### Inertia + +Inertia is a metric used to assess the quality of the clusters formed. It is the sum of squared distances of samples to their nearest cluster center. + +## Types of Clustering + +1. **Hard Clustering**: Each data point either belongs to a cluster completely or not at all. +2. **Soft Clustering (Fuzzy Clustering)**: Each data point can belong to multiple clusters with varying degrees of membership. + +## Clustering Algorithms + +### K-Means Clustering + +K-Means is a popular clustering algorithm that partitions the data into k clusters, where each data point belongs to the cluster with the nearest mean. The algorithm follows these steps: +1. Initialize k centroids randomly. +2. Assign each data point to the nearest centroid. +3. Recalculate the centroids as the mean of all data points assigned to each cluster. +4. Repeat steps 2 and 3 until convergence. + +### Hierarchical Clustering + +Hierarchical clustering builds a tree of clusters. There are two types: +- **Agglomerative (bottom-up)**: Starts with each data point as a separate cluster and merges the closest pairs of clusters iteratively. +- **Divisive (top-down)**: Starts with all data points in one cluster and splits the cluster iteratively into smaller clusters. + +### DBSCAN (Density-Based Spatial Clustering of Applications with Noise) + +DBSCAN groups together points that are close to each other based on a distance measurement and a minimum number of points. It can find arbitrarily shaped clusters and is robust to noise. + +## Implementation + +### Using Scikit-learn + +Scikit-learn is a popular machine learning library in Python that provides tools for clustering. + +### Code Example + +```python +import numpy as np +import pandas as pd +from sklearn.cluster import KMeans +from sklearn.preprocessing import StandardScaler +from sklearn.metrics import silhouette_score + +# Load dataset +data = pd.read_csv('path/to/your/dataset.csv') + +# Preprocess the data +scaler = StandardScaler() +data_scaled = scaler.fit_transform(data) + +# Initialize and fit KMeans model +kmeans = KMeans(n_clusters=3, random_state=42) +kmeans.fit(data_scaled) + +# Get cluster labels +labels = kmeans.labels_ + +# Calculate silhouette score +silhouette_avg = silhouette_score(data_scaled, labels) +print("Silhouette Score:", silhouette_avg) + +# Add cluster labels to the original data +data['Cluster'] = labels + +print(data.head()) +``` + +## Evaluation Metrics + +- **Silhouette Score**: Measures how similar a data point is to its own cluster compared to other clusters. +- **Inertia (Within-cluster Sum of Squares)**: Measures the compactness of the clusters. +- **Davies-Bouldin Index**: Measures the average similarity ratio of each cluster with the cluster that is most similar to it. +- **Dunn Index**: Ratio of the minimum inter-cluster distance to the maximum intra-cluster distance. + +## Conclusion + +Clustering is a powerful technique for discovering structure in data. Understanding different clustering algorithms and their evaluation metrics is crucial for selecting the appropriate method for a given problem. diff --git a/contrib/machine-learning/cost-functions.md b/contrib/machine-learning/cost-functions.md new file mode 100644 index 00000000..c1fe2170 --- /dev/null +++ b/contrib/machine-learning/cost-functions.md @@ -0,0 +1,235 @@ + +# Cost Functions in Machine Learning + +Cost functions, also known as loss functions, play a crucial role in training machine learning models. They measure how well the model performs on the training data by quantifying the difference between predicted and actual values. Different types of cost functions are used depending on the problem domain and the nature of the data. + +## Types of Cost Functions + +### 1. Mean Squared Error (MSE) + +**Explanation:** +MSE is one of the most commonly used cost functions, particularly in regression problems. It calculates the average squared difference between the predicted and actual values. + +**Mathematical Formulation:** +The MSE is defined as: +$$MSE = \frac{1}{n} \sum_{i=1}^{n} (y_i - \hat{y}_i)^2$$ +Where: +- `n` is the number of samples. +- $y_i$ is the actual value. +- $\hat{y}_i$ is the predicted value. + +**Advantages:** +- Sensitive to large errors due to squaring. +- Differentiable and convex, facilitating optimization. + +**Disadvantages:** +- Sensitive to outliers, as the squared term amplifies their impact. + +**Python Implementation:** +```python +import numpy as np + +def mean_squared_error(y_true, y_pred): + n = len(y_true) + return np.mean((y_true - y_pred) ** 2) +``` + +### 2. Mean Absolute Error (MAE) + +**Explanation:** +MAE is another commonly used cost function for regression tasks. It measures the average absolute difference between predicted and actual values. + +**Mathematical Formulation:** +The MAE is defined as: +$$MAE = \frac{1}{n} \sum_{i=1}^{n} |y_i - \hat{y}_i|$$ +Where: +- `n` is the number of samples. +- $y_i$ is the actual value. +- $\hat{y}_i$ is the predicted value. + +**Advantages:** +- Less sensitive to outliers compared to MSE. +- Provides a linear error term, which can be easier to interpret. + + +**Disadvantages:** +- Not differentiable at zero, which can complicate optimization. + +**Python Implementation:** +```python +import numpy as np + +def mean_absolute_error(y_true, y_pred): + n = len(y_true) + return np.mean(np.abs(y_true - y_pred)) +``` + +### 3. Cross-Entropy Loss (Binary) + +**Explanation:** +Cross-entropy loss is commonly used in binary classification problems. It measures the dissimilarity between the true and predicted probability distributions. + +**Mathematical Formulation:** + +For binary classification, the cross-entropy loss is defined as: + +$$\text{Cross-Entropy} = -\frac{1}{n} \sum_{i=1}^{n} [y_i \log(\hat{y}_i) + (1 - y_i) \log(1 - \hat{y}_i)]$$ + +Where: +- `n` is the number of samples. +- $y_i$ is the actual class label (0 or 1). +- $\hat{y}_i$ is the predicted probability of the positive class. + + +**Advantages:** +- Penalizes confident wrong predictions heavily. +- Suitable for probabilistic outputs. + +**Disadvantages:** +- Sensitive to class imbalance. + +**Python Implementation:** +```python +import numpy as np + +def binary_cross_entropy(y_true, y_pred): + n = len(y_true) + return -np.mean(y_true * np.log(y_pred) + (1 - y_true) * np.log(1 - y_pred)) +``` + +### 4. Cross-Entropy Loss (Multiclass) + +**Explanation:** +For multiclass classification problems, the cross-entropy loss is adapted to handle multiple classes. + +**Mathematical Formulation:** + +The multiclass cross-entropy loss is defined as: + +$$\text{Cross-Entropy} = -\frac{1}{n} \sum_{i=1}^{n} \sum_{c=1}^{C} y_{i,c} \log(\hat{y}_{i,c})$$ + +Where: +- `n` is the number of samples. +- `C` is the number of classes. +- $y_{i,c}$ is the indicator function for the true class of sample `i`. +- $\hat{y}_{i,c}$ is the predicted probability of sample `i` belonging to class `c`. + +**Advantages:** +- Handles multiple classes effectively. +- Encourages the model to assign high probabilities to the correct classes. + +**Disadvantages:** +- Requires one-hot encoding for class labels, which can increase computational complexity. + +**Python Implementation:** +```python +import numpy as np + +def categorical_cross_entropy(y_true, y_pred): + n = len(y_true) + return -np.mean(np.sum(y_true * np.log(y_pred), axis=1)) +``` + +### 5. Hinge Loss (SVM) + +**Explanation:** +Hinge loss is commonly used in support vector machines (SVMs) for binary classification tasks. It penalizes misclassifications by a linear margin. + +**Mathematical Formulation:** + +For binary classification, the hinge loss is defined as: + +$$\text{Hinge Loss} = \frac{1}{n} \sum_{i=1}^{n} \max(0, 1 - y_i \cdot \hat{y}_i)$$ + +Where: +- `n` is the number of samples. +- $y_i$ is the actual class label (-1 or 1). +- $\hat{y}_i$ is the predicted score for sample \( i \). + +**Advantages:** +- Encourages margin maximization in SVMs. +- Robust to outliers due to the linear penalty. + +**Disadvantages:** +- Not differentiable at the margin, which can complicate optimization. + +**Python Implementation:** +```python +import numpy as np + +def hinge_loss(y_true, y_pred): + n = len(y_true) + loss = np.maximum(0, 1 - y_true * y_pred) + return np.mean(loss) +``` + +### 6. Huber Loss + +**Explanation:** +Huber loss is a combination of MSE and MAE, providing a compromise between the two. It is less sensitive to outliers than MSE and provides a smooth transition to MAE for large errors. + +**Mathematical Formulation:** + +The Huber loss is defined as: + + +$$\text{Huber Loss} = \frac{1}{n} \sum_{i=1}^{n} \left\{ +\begin{array}{ll} +\frac{1}{2} (y_i - \hat{y}_i)^2 & \text{if } |y_i - \hat{y}_i| \leq \delta \\ +\delta(|y_i - \hat{y}_i| - \frac{1}{2} \delta) & \text{otherwise} +\end{array} +\right.$$ + +Where: +- `n` is the number of samples. +- $\delta$ is a threshold parameter. + +**Advantages:** +- Provides a smooth loss function. +- Less sensitive to outliers than MSE. + +**Disadvantages:** +- Requires tuning of the threshold parameter. + +**Python Implementation:** +```python +import numpy as np + +def huber_loss(y_true, y_pred, delta): + error = y_true - y_pred + loss = np.where(np.abs(error) <= delta, 0.5 * error ** 2, delta * (np.abs(error) - 0.5 * delta)) + return np.mean(loss) +``` + +### 7. Log-Cosh Loss + +**Explanation:** +Log-Cosh loss is a smooth approximation of the MAE and is less sensitive to outliers than MSE. It provides a smooth transition from quadratic for small errors to linear for large errors. + +**Mathematical Formulation:** + +The Log-Cosh loss is defined as: + +$$\text{Log-Cosh Loss} = \frac{1}{n} \sum_{i=1}^{n} \log(\cosh(y_i - \hat{y}_i))$$ + +Where: +- `n` is the number of samples. + +**Advantages:** +- Smooth and differentiable everywhere. +- Less sensitive to outliers. + +**Disadvantages:** +- Computationally more expensive than simple losses like MSE. + +**Python Implementation:** +```python +import numpy as np + +def logcosh_loss(y_true, y_pred): + error = y_true - y_pred + loss = np.log(np.cosh(error)) + return np.mean(loss) +``` + +These implementations provide various options for cost functions suitable for different machine learning tasks. Each function has its advantages and disadvantages, making them suitable for different scenarios and problem domains. diff --git a/contrib/machine-learning/Decision-Tree.md b/contrib/machine-learning/decision-tree.md similarity index 99% rename from contrib/machine-learning/Decision-Tree.md rename to contrib/machine-learning/decision-tree.md index 6563a223..8159bcf2 100644 --- a/contrib/machine-learning/Decision-Tree.md +++ b/contrib/machine-learning/decision-tree.md @@ -254,4 +254,4 @@ The final decision tree classifies instances based on the following rules: - If Outlook is Rain and Wind is Weak, PlayTennis is Yes - If Outlook is Rain and Wind is Strong, PlayTennis is No -> Note that the calculated entropies and information gains may vary slightly depending on the specific implementation and rounding methods used. \ No newline at end of file +> Note that the calculated entropies and information gains may vary slightly depending on the specific implementation and rounding methods used. diff --git a/contrib/machine-learning/eda.md b/contrib/machine-learning/eda.md new file mode 100644 index 00000000..1559a099 --- /dev/null +++ b/contrib/machine-learning/eda.md @@ -0,0 +1,184 @@ +# Exploratory Data Analysis + +Exploratory Data Analysis (EDA) is an approach to analyzing data sets to summarize their main characteristics, often with visual methods. EDA is used to understand the data, get a sense of the data, and to identify relationships between variables. EDA is a crucial step in the data analysis process and should be done before building a model. + +## Why is EDA important? + +1. **Understand the data**: EDA helps to understand the data, its structure, and its characteristics. + +2. **Identify patterns and relationships**: EDA helps to identify patterns and relationships between variables. + +3. **Detect outliers and anomalies**: EDA helps to detect outliers and anomalies in the data. + +4. **Prepare data for modeling**: EDA helps to prepare the data for modeling by identifying missing values, handling missing values, and transforming variables. + +## Steps in EDA + +1. **Data Collection**: Collect the data from various sources. + +2. **Data Cleaning**: Clean the data by handling missing values, removing duplicates, and transforming variables. + +3. **Data Exploration**: Explore the data by visualizing the data, summarizing the data, and identifying patterns and relationships. + +4. **Data Analysis**: Analyze the data by performing statistical analysis, hypothesis testing, and building models. + +5. **Data Visualization**: Visualize the data using various plots and charts to understand the data better. + +## Tools for EDA + +1. **Python**: Python is a popular programming language for data analysis and has many libraries for EDA, such as Pandas, NumPy, Matplotlib, Seaborn, and Plotly. + +2. **Jupiter Notebook**: Jupyter Notebook is an open-source web application that allows you to create and share documents that contain live code, equations, visualizations, and narrative text. + +## Techniques for EDA + +1. **Descriptive Statistics**: Descriptive statistics summarize the main characteristics of a data set, such as mean, median, mode, standard deviation, and variance. + +2. **Data Visualization**: Data visualization is the graphical representation of data to understand the data better, such as histograms, scatter plots, box plots, and heat maps. + +3. **Correlation Analysis**: Correlation analysis is used to measure the strength and direction of the relationship between two variables. + +4. **Hypothesis Testing**: Hypothesis testing is used to test a hypothesis about a population parameter based on sample data. + +5. **Dimensionality Reduction**: Dimensionality reduction is the process of reducing the number of variables in a data set while retaining as much information as possible. + +6. **Clustering Analysis**: Clustering analysis is used to group similar data points together based on their characteristics. + +## Commonly Used Techniques in EDA + +1. **Uni-variate Analysis**: Uni-variate analysis is the simplest form of data analysis that involves analyzing a single variable at a time. + +2. **Bi-variate Analysis**: Bi-variate analysis involves analyzing two variables at a time to understand the relationship between them. + +3. **Multi-variate Analysis**: Multi-variate analysis involves analyzing more than two variables at a time to understand the relationship between them. + +## Understand with an Example + +Let's understand EDA with an example. Here we use a famous dataset called Iris dataset. + +The dataset consists of 150 samples of iris flowers, where each sample represents measurements of four features (variables) for three species of iris flowers. + +The four features measured are : +Sepal length (in cm) Sepal width (in cm) Petal length (in cm) Petal width (in cm). + +The three species of iris flowers included in the dataset are : +**Setosa**, **Versicolor**, **Virginica** + +```python +# Import libraries +import pandas as pd +import numpy as np +import matplotlib.pyplot as plt +import seaborn as sns +from sklearn import datasets + +# Load the Iris dataset +iris = datasets.load_iris() +df = pd.DataFrame(iris.data, columns=iris.feature_names) +df.head() +``` + +| Sepal Length (cm) | Sepal Width (cm) | Petal Length (cm) | Petal Width (cm) | +|-------------------|------------------|-------------------|------------------| +| 5.1 | 3.5 | 1.4 | 0.2 | +| 4.9 | 3.0 | 1.4 | 0.2 | +| 4.7 | 3.2 | 1.3 | 0.2 | +| 4.6 | 3.1 | 1.5 | 0.2 | +| 5.0 | 3.6 | 1.4 | 0.2 | + + +### Uni-variate Analysis + +```python +# Uni-variate Analysis +df_setosa=df.loc[df['species']=='setosa'] +df_virginica=df.loc[df['species']=='virginica'] +df_versicolor=df.loc[df['species']=='versicolor'] + +plt.plot(df_setosa['sepal_length']) +plt.plot(df_virginica['sepal_length']) +plt.plot(df_versicolor['sepal_length']) +plt.xlabel('sepal length') +plt.show() +``` + + +```python +plt.hist(df_setosa['petal_length']) +plt.hist(df_virginica['petal_length']) +plt.hist(df_versicolor['petal_length']) +plt.xlabel('petal length') +plt.show() +``` + + +### Bi-variate Analysis + +```python +# Bi-variate Analysis +sns.FacetGrid(df,hue="species",height=5).map(plt.scatter,"petal_length","sepal_width").add_legen() +plt.show() +``` + + +### Multi-variate Analysis + +```python +# Multi-variate Analysis +sns.pairplot(df,hue="species",height=3) +``` + + +### Correlation Analysis + +```python +# Correlation Analysis +corr_matrix = df.corr() +sns.heatmap(corr_matrix) +``` +| | sepal_length | sepal_width | petal_length | petal_width | +|-------------|--------------|-------------|--------------|-------------| +| sepal_length| 1.000000 | -0.109369 | 0.871754 | 0.817954 | +| sepal_width | -0.109369 | 1.000000 | -0.420516 | -0.356544 | +| petal_length| 0.871754 | -0.420516 | 1.000000 | 0.962757 | +| petal_width | 0.817954 | -0.356544 | 0.962757 | 1.000000 | + + + +## Exploratory Data Analysis (EDA) Report on Iris Dataset + +### Introduction +The Iris dataset consists of 150 samples of iris flowers, each characterized by four features: Sepal Length, Sepal Width, Petal Length, and Petal Width. These samples belong to three species of iris flowers: Setosa, Versicolor, and Virginica. In this EDA report, we explore the dataset to gain insights into the characteristics and relationships among the features and species. + +### Uni-variate Analysis +Uni-variate analysis examines each variable individually. +- Sepal Length: The distribution of Sepal Length varies among the different species, with Setosa generally having shorter sepals compared to Versicolor and Virginica. +- Petal Length: Setosa tends to have shorter petal lengths, while Versicolor and Virginica have relatively longer petal lengths. + +### Bi-variate Analysis +Bi-variate analysis explores the relationship between two variables. +- Petal Length vs. Sepal Width: There is a noticeable separation between species, especially Setosa, which typically has shorter and wider sepals compared to Versicolor and Virginica. +- This analysis suggests potential patterns distinguishing the species based on these two features. + +### Multi-variate Analysis +Multi-variate analysis considers interactions among multiple variables simultaneously. +- Pairplot: The pairplot reveals distinctive clusters for each species, particularly in the combinations of Petal Length and Petal Width, indicating clear separation among species based on these features. + +### Correlation Analysis +Correlation analysis examines the relationship between variables. +- Correlation Heatmap: There are strong positive correlations between Petal Length and Petal Width, as well as between Petal Length and Sepal Length. Sepal Width shows a weaker negative correlation with Petal Length and Petal Width. + +### Insights +1. Petal dimensions (length and width) exhibit strong correlations, suggesting that they may collectively contribute more significantly to distinguishing between iris species. +2. Setosa tends to have shorter and wider sepals compared to Versicolor and Virginica. +3. The combination of Petal Length and Petal Width appears to be a more effective discriminator among iris species, as indicated by the distinct clusters observed in multi-variate analysis. + +### Conclusion +Through comprehensive exploratory data analysis, we have gained valuable insights into the Iris dataset, highlighting key characteristics and relationships among features and species. Further analysis and modeling could leverage these insights to develop robust classification models for predicting iris species based on their measurements. + +## Conclusion + +Exploratory Data Analysis (EDA) is a critical step in the data analysis process that helps to understand the data, identify patterns and relationships, detect outliers, and prepare the data for modeling. By using various techniques and tools, such as descriptive statistics, data visualization, correlation analysis, and hypothesis testing, EDA provides valuable insights into the data, enabling data scientists to make informed decisions and build accurate models. + + + diff --git a/contrib/machine-learning/ensemble-learning.md b/contrib/machine-learning/ensemble-learning.md new file mode 100644 index 00000000..508f45e7 --- /dev/null +++ b/contrib/machine-learning/ensemble-learning.md @@ -0,0 +1,140 @@ +# Ensemble Learning + +Ensemble Learning is a powerful machine learning paradigm that combines multiple models to achieve better performance than any individual model. The idea is to leverage the strengths of different models to improve overall accuracy, robustness, and generalization. + + + +## Introduction + +Ensemble Learning is a technique that combines the predictions from multiple machine learning models to make more accurate and robust predictions than a single model. It leverages the diversity of different models to reduce errors and improve performance. + +## Types of Ensemble Learning + +### Bagging + +Bagging, or Bootstrap Aggregating, involves training multiple versions of the same model on different subsets of the training data and averaging their predictions. The most common example of bagging is the `RandomForest` algorithm. + +### Boosting + +Boosting focuses on training models sequentially, where each new model corrects the errors made by the previous ones. This way, the ensemble learns from its mistakes, leading to improved performance. `AdaBoost` and `Gradient Boosting` are popular examples of boosting algorithms. + +### Stacking + +Stacking involves training multiple models (the base learners) and a meta-model that combines their predictions. The base learners are trained on the original dataset, while the meta-model is trained on the outputs of the base learners. This approach allows leveraging the strengths of different models. + +## Advantages and Disadvantages + +### Advantages + +- **Improved Accuracy**: Combines the strengths of multiple models. +- **Robustness**: Reduces the risk of overfitting and model bias. +- **Versatility**: Can be applied to various machine learning tasks, including classification and regression. + +### Disadvantages + +- **Complexity**: More complex than individual models, making interpretation harder. +- **Computational Cost**: Requires more computational resources and training time. +- **Implementation**: Can be challenging to implement and tune effectively. + +## Key Concepts + +- **Diversity**: The models in the ensemble should be diverse to benefit from their different strengths. +- **Voting/Averaging**: For classification, majority voting is used to combine predictions. For regression, averaging is used. +- **Weighting**: In some ensembles, models are weighted based on their accuracy or other metrics. + +## Code Examples + +### Bagging with Random Forest + +Below is an example of using Random Forest for classification on the Iris dataset. + +```python +import numpy as np +import pandas as pd +from sklearn.datasets import load_iris +from sklearn.ensemble import RandomForestClassifier +from sklearn.model_selection import train_test_split +from sklearn.metrics import accuracy_score, classification_report + +# Load dataset +iris = load_iris() +X, y = iris.data, iris.target + +# Split dataset +X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.3, random_state=42) + +# Initialize Random Forest model +clf = RandomForestClassifier(n_estimators=100, random_state=42) + +# Train the model +clf.fit(X_train, y_train) + +# Make predictions +y_pred = clf.predict(X_test) + +# Evaluate the model +accuracy = accuracy_score(y_test, y_pred) +print(f"Accuracy: {accuracy * 100:.2f}%") +print("Classification Report:\n", classification_report(y_test, y_pred)) +``` + +### Boosting with AdaBoost +Below is an example of using AdaBoost for classification on the Iris dataset. + +``` +from sklearn.ensemble import AdaBoostClassifier +from sklearn.tree import DecisionTreeClassifier + +# Initialize base model +base_model = DecisionTreeClassifier(max_depth=1) + +# Initialize AdaBoost model +ada_clf = AdaBoostClassifier(base_estimator=base_model, n_estimators=50, random_state=42) + +# Train the model +ada_clf.fit(X_train, y_train) + +# Make predictions +y_pred = ada_clf.predict(X_test) + +# Evaluate the model +accuracy = accuracy_score(y_test, y_pred) +print(f"Accuracy: {accuracy * 100:.2f}%") +print("Classification Report:\n", classification_report(y_test, y_pred)) +``` + +### Stacking with Multiple Models +Below is an example of using stacking with multiple models for classification on the Iris dataset. + +``` +from sklearn.linear_model import LogisticRegression +from sklearn.neighbors import KNeighborsClassifier +from sklearn.svm import SVC +from sklearn.ensemble import StackingClassifier + +# Define base models +base_models = [ + ('knn', KNeighborsClassifier(n_neighbors=5)), + ('svc', SVC(kernel='linear', probability=True)) +] + +# Define meta-model +meta_model = LogisticRegression() + +# Initialize Stacking model +stacking_clf = StackingClassifier(estimators=base_models, final_estimator=meta_model, cv=5) + +# Train the model +stacking_clf.fit(X_train, y_train) + +# Make predictions +y_pred = stacking_clf.predict(X_test) + +# Evaluate the model +accuracy = accuracy_score(y_test, y_pred) +print(f"Accuracy: {accuracy * 100:.2f}%") +print("Classification Report:\n", classification_report(y_test, y_pred)) +``` + +## Conclusion +Ensemble Learning is a powerful technique that combines multiple models to improve overall performance. By leveraging the strengths of different models, it provides better accuracy, robustness, and generalization. However, it comes with increased complexity and computational cost. Understanding and implementing ensemble methods can significantly enhance machine learning solutions. diff --git a/contrib/machine-learning/hierarchical-clustering.md b/contrib/machine-learning/hierarchical-clustering.md new file mode 100644 index 00000000..93822703 --- /dev/null +++ b/contrib/machine-learning/hierarchical-clustering.md @@ -0,0 +1,99 @@ +# Hierarchical Clustering + +Hierarchical Clustering is a method of cluster analysis that seeks to build a hierarchy of clusters. This README provides an overview of the hierarchical clustering algorithm, including its fundamental concepts, types, steps, and how to implement it using Python. + +## Introduction + +Hierarchical Clustering is an unsupervised learning method used to group similar objects into clusters. Unlike other clustering techniques, hierarchical clustering does not require the number of clusters to be specified beforehand. It produces a tree-like structure called a dendrogram, which displays the arrangement of the clusters and their sub-clusters. + +## Concepts + +### Dendrogram + +A dendrogram is a tree-like diagram that records the sequences of merges or splits. It is a useful tool for visualizing the process of hierarchical clustering. + +### Distance Measure + +Distance measures are used to quantify the similarity or dissimilarity between data points. Common distance measures include Euclidean distance, Manhattan distance, and cosine similarity. + +### Linkage Criteria + +Linkage criteria determine how the distance between clusters is calculated. Different linkage criteria include single linkage, complete linkage, average linkage, and Ward's linkage. + +## Types of Hierarchical Clustering + +1. **Agglomerative Clustering (Bottom-Up Approach)**: + - Starts with each data point as a separate cluster. + - Repeatedly merges the closest pairs of clusters until only one cluster remains or a stopping criterion is met. + +2. **Divisive Clustering (Top-Down Approach)**: + - Starts with all data points in a single cluster. + - Repeatedly splits clusters into smaller clusters until each data point is its own cluster or a stopping criterion is met. + +## Steps in Hierarchical Clustering + +1. **Calculate Distance Matrix**: Compute the distance between each pair of data points. +2. **Create Clusters**: Treat each data point as a single cluster. +3. **Merge Closest Clusters**: Find the two clusters that are closest to each other and merge them into a single cluster. +4. **Update Distance Matrix**: Update the distance matrix to reflect the distance between the new cluster and the remaining clusters. +5. **Repeat**: Repeat steps 3 and 4 until all data points are merged into a single cluster or the desired number of clusters is achieved. + +## Linkage Criteria + +1. **Single Linkage (Minimum Linkage)**: The distance between two clusters is defined as the minimum distance between any single data point in the first cluster and any single data point in the second cluster. +2. **Complete Linkage (Maximum Linkage)**: The distance between two clusters is defined as the maximum distance between any single data point in the first cluster and any single data point in the second cluster. +3. **Average Linkage**: The distance between two clusters is defined as the average distance between all pairs of data points, one from each cluster. +4. **Ward's Linkage**: The distance between two clusters is defined as the increase in the sum of squared deviations from the mean when the two clusters are merged. + +## Implementation + +### Using Scikit-learn + +Scikit-learn is a popular machine learning library in Python that provides tools for hierarchical clustering. + +### Code Example + +```python +import numpy as np +import pandas as pd +import matplotlib.pyplot as plt +from scipy.cluster.hierarchy import dendrogram, linkage +from sklearn.cluster import AgglomerativeClustering +from sklearn.preprocessing import StandardScaler + +# Load dataset +data = pd.read_csv('path/to/your/dataset.csv') + +# Preprocess the data +scaler = StandardScaler() +data_scaled = scaler.fit_transform(data) + +# Perform hierarchical clustering +Z = linkage(data_scaled, method='ward') + +# Plot the dendrogram +plt.figure(figsize=(10, 7)) +dendrogram(Z) +plt.title('Dendrogram') +plt.xlabel('Data Points') +plt.ylabel('Distance') +plt.show() + +# Perform Agglomerative Clustering +agg_clustering = AgglomerativeClustering(n_clusters=3, affinity='euclidean', linkage='ward') +labels = agg_clustering.fit_predict(data_scaled) + +# Add cluster labels to the original data +data['Cluster'] = labels +print(data.head()) +``` + +## Evaluation Metrics + +- **Silhouette Score**: Measures how similar a data point is to its own cluster compared to other clusters. +- **Cophenetic Correlation Coefficient**: Measures how faithfully a dendrogram preserves the pairwise distances between the original data points. +- **Dunn Index**: Ratio of the minimum inter-cluster distance to the maximum intra-cluster distance. + +## Conclusion + +Hierarchical clustering is a versatile and intuitive method for clustering data. It is particularly useful when the number of clusters is not known beforehand. By understanding the different linkage criteria and evaluation metrics, one can effectively apply hierarchical clustering to various types of data. diff --git a/contrib/machine-learning/index.md b/contrib/machine-learning/index.md index e3a8f0b6..7ee61ebe 100644 --- a/contrib/machine-learning/index.md +++ b/contrib/machine-learning/index.md @@ -1,13 +1,30 @@ # List of sections -- [Binomial Distribution](binomial_distribution.md) -- [Regression in Machine Learning](Regression.md) +- [Introduction to scikit-learn](sklearn-introduction.md) +- [Binomial Distribution](binomial-distribution.md) +- [Naive Bayes](naive-bayes.md) +- [Regression in Machine Learning](regression.md) +- [Polynomial Regression](polynomial-regression.md) - [Confusion Matrix](confusion-matrix.md) -- [Decision Tree Learning](Decision-Tree.md) +- [Decision Tree Learning](decision-tree.md) +- [Random Forest](random-forest.md) - [Support Vector Machine Algorithm](support-vector-machine.md) -- [Artificial Neural Network from the Ground Up](ArtificialNeuralNetwork.md) -- [TensorFlow.md](tensorFlow.md) -- [PyTorch.md](pytorch.md) -- [Types of optimizers](Types_of_optimizers.md) +- [Ensemble Learning](ensemble-learning.md) +- [Types of optimizers](types-of-optimizers.md) - [Logistic Regression](logistic-regression.md) +- [Types_of_Cost_Functions](cost-functions.md) +- [Clustering](clustering.md) +- [Hierarchical Clustering](hierarchical-clustering.md) - [Grid Search](grid-search.md) +- [K-Means](kmeans.md) +- [K-nearest neighbor (KNN)](knn.md) +- [Xgboost](xgboost.md) +- [Artificial Neural Network from the Ground Up](ann.md) +- [Introduction To Convolutional Neural Networks (CNNs)](intro-to-cnn.md) +- [TensorFlow](tensorflow.md) +- [PyTorch](pytorch.md) +- [PyTorch Fundamentals](pytorch-fundamentals.md) +- [Transformers](transformers.md) +- [Reinforcement Learning](reinforcement-learning.md) +- [Neural network regression](neural-network-regression.md) +- [Exploratory Data Analysis](eda.md) diff --git a/contrib/machine-learning/intro-to-cnn.md b/contrib/machine-learning/intro-to-cnn.md new file mode 100644 index 00000000..0221ca10 --- /dev/null +++ b/contrib/machine-learning/intro-to-cnn.md @@ -0,0 +1,225 @@ +# Understanding Convolutional Neural Networks (CNN) + +## Introduction +Convolutional Neural Networks (CNNs) are a specialized type of artificial neural network designed primarily for processing structured grid data like images. CNNs are particularly powerful for tasks involving image recognition, classification, and computer vision. They have revolutionized these fields, outperforming traditional neural networks by leveraging their unique architecture to capture spatial hierarchies in images. + +### Why CNNs are Superior to Traditional Neural Networks +1. **Localized Receptive Fields**: CNNs use convolutional layers that apply filters to local regions of the input image. This localized connectivity ensures that the network learns spatial hierarchies and patterns, such as edges and textures, which are essential for image recognition tasks. +2. **Parameter Sharing**: In CNNs, the same filter (set of weights) is used across different parts of the input, significantly reducing the number of parameters compared to fully connected layers in traditional neural networks. This not only lowers the computational cost but also mitigates the risk of overfitting. +3. **Translation Invariance**: Due to the shared weights and pooling operations, CNNs are inherently invariant to translations of the input image. This means that they can recognize objects even when they appear in different locations within the image. +4. **Hierarchical Feature Learning**: CNNs automatically learn a hierarchy of features from low-level features like edges to high-level features like shapes and objects. Traditional neural networks, on the other hand, require manual feature extraction which is less effective and more time-consuming. + +### Use Cases of CNNs +- **Image Classification**: Identifying objects within an image (e.g., classifying a picture as containing a cat or a dog). +- **Object Detection**: Detecting and locating objects within an image (e.g., finding faces in a photo). +- **Image Segmentation**: Partitioning an image into segments or regions (e.g., dividing an image into different objects and background). +- **Medical Imaging**: Analyzing medical scans like MRI, CT, and X-rays for diagnosis. + +> This guide will walk you through the fundamentals of CNNs and their implementation in Python. We'll build a simple CNN from scratch, explaining each component to help you understand how CNNs process images and extract features. + +### Let's start by understanding the basic architecture of CNNs. + +## CNN Architecture +Convolution layers, pooling layers, and fully connected layers are just a few of the many building blocks that CNNs use to automatically and adaptively learn spatial hierarchies of information through backpropagation. + +### Convolutional Layer +The convolutional layer is the core building block of a CNN. The layer's parameters consist of a set of learnable filters (or kernels), which have a small receptive field but extend through the full depth of the input volume. + +#### Input Shape +The dimensions of the input image, including the number of channels (e.g., 3 for RGB images & 1 for Grayscale images). + + +- The input matrix is a binary image of handwritten digits, +where '1' marks the pixels containing the digit (ink/grayscale area) and '0' marks the background pixels (empty space). +- The first matrix shows the represnetation of 1 and 0, which can be depicted as a vertical line and a closed loop. +- The second matrix represents 9, combining the loop and line. + +#### Strides +The step size with which the filter moves across the input image. + + +- This visualization will help you understand how the filter (kernel) moves acroos the input matrix with stride values of (3,3) and (2,2). +- A stride of 1 means the filter moves one step at a time, ensuring it covers the entire input matrix. +- However, with larger strides (like 3 or 2 in this example), the filter may not cover all elements, potentially missing some information. +- While this might seem like a drawback, higher strides are often used to reduce computational cost and decrease the output size, which can be beneficial in speeding up the training process and preventing overfitting. + +#### Padding +Determines whether the output size is the same as the input size ('same') or reduced ('valid'). + + +- `Same` padding is preferred in earlier layers to preserve spatial and edge information, as it can help the network learn more detailed features. +- Choose `valid` padding when focusing on the central input region or requiring specific output dimensions. +- Padding value can be determined by $ ( f - 1 ) \over 2 $, where f isfilter size + +#### Filters +Small matrices that slide over the input data to extract features. + + +- The first filter aims to detect closed loops within the input image, being highly relevant for recognizing digits with circular or oval shapes, such as '0', '6', '8', or '9'. +- The next filter helps in detecting vertical lines, crucial for identifying digits like '1', '4', '7', and parts of other digits that contain vertical strokes. +- The last filter shows how to detect diagonal lines in the input image, useful for identifying the slashes present in digits like '1', '7', or parts of '4' and '9'. + +#### Output +A set of feature maps that represent the presence of different features in the input. + + +- With no padding and a stride of 1, the 3x3 filter moves one step at a time across the 7x5 input matrix. The filter can only move within the original boundaries of the input, resulting in a smaller 5x3 output matrix. This configuration is useful when you want to reduce the spatial dimensions of the feature map while preserving the exact spatial relationships between features. +- By adding zero padding to the input matrix, it is expanded to 9x7, allowing the 3x3 filter to "fit" fully on the edges and corners. With a stride of 1, the filter still moves one step at a time, but now the output matrix is the same size (7x5) as the original input. Same padding is often preferred in early layers of a CNN to preserve spatial information and avoid rapid feature map shrinkage. +- Without padding, the 3x3 filter operates within the original input matrix boundaries, but now it moves two steps at a time (stride 2). This significantly reduces the output matrix size to 3x2. Larger strides are employed to decrease computational cost and the output size, which can be beneficial in speeding up the training process and preventing overfitting. However, they might miss some finer details due to the larger jumps. +- The output dimension of a CNN model is given by, $$ n_{out} = { n_{in} + (2 \cdot p) - k \over s } $$ +where, + nin = number of input features + p = padding + k = kernel size + s = stride + +- Also, the number of trainable parameters for each layer is given by, $ (n_c \cdot [k \cdot k] \cdot f) + f $ +where, + nc = number of input channels + k x k = kernel size + f = number of filters + an additional f is added for bias + +### Pooling Layer +Pooling layers reduce the dimensionality of each feature map while retaining the most critical information. The most common form of pooling is max pooling. +- **Input Shape:** The dimensions of the feature map from the convolutional layer. +- **Pooling Size:** The size of the pooling window (e.g., 2x2). +- **Strides:** The step size for the pooling operation. +- **Output:** A reduced feature map highlighting the most important features. +
+ 
+
+
+- The high values (8) indicate that the "closed loop" filter found a strong match in those regions.
+- First matrix of size 6x4 represents a downsampled version of the input.
+- While the second matrix with 3x2, resulting in more aggressive downsampling.
+
+### Flatten Layer
+The flatten layer converts the 2D matrix data to a 1D vector, which can be fed into a fully connected (dense) layer.
+- **Input Shape:** The 2D feature maps from the previous layer.
+- **Output:** A 1D vector that represents the same data in a flattened format.
+
+
+### Dropout Layer
+Dropout is a regularization technique to prevent overfitting in neural networks by randomly setting a fraction of input units to zero at each update during training time.
+- **Input Shape:** The data from the previous layer.
+- **Dropout Rate:** The fraction of units to drop (e.g., 0.5 for 50% dropout).
+- **Output:** The same shape as the input, with some units set to zero.
+
+
+- The updated 0 values represents the dropped units.
+
+## Implementation
+
+Below is the implementation of a simple CNN in Python. Each function within the `CNN` class corresponds to a layer in the network.
+
+```python
+import numpy as np
+
+class CNN:
+ def __init__(self):
+ pass
+
+ def convLayer(self, input_shape, channels, strides, padding, filter_size):
+ height, width = input_shape
+ input_shape_with_channels = (height, width, channels)
+ print("Input Shape (with channels):", input_shape_with_channels)
+
+ # Generate random input and filter matrices
+ input_matrix = np.random.randint(0, 10, size=input_shape_with_channels)
+ filter_matrix = np.random.randint(0, 5, size=(filter_size[0], filter_size[1], channels))
+
+ print("\nInput Matrix:\n", input_matrix[:, :, 0])
+ print("\nFilter Matrix:\n", filter_matrix[:, :, 0])
+
+ padding = padding.lower()
+
+ if padding == 'same':
+ # Calculate padding needed for each dimension
+ pad_height = filter_size[0] // 2
+ pad_width = filter_size[1] // 2
+
+ # Apply padding to the input matrix
+ input_matrix = np.pad(input_matrix, ((pad_height, pad_height), (pad_width, pad_width), (0, 0)), mode='constant')
+
+ # Adjust height and width to consider the padding
+ height += 2 * pad_height
+ width += 2 * pad_width
+
+ elif padding == 'valid':
+ pass
+
+ else:
+ return "Invalid Padding!!"
+
+ # Output dimensions
+ conv_height = (height - filter_size[0]) // strides[0] + 1
+ conv_width = (width - filter_size[1]) // strides[1] + 1
+ output_matrix = np.zeros((conv_height, conv_width, channels))
+
+ # Convolution Operation
+ for i in range(0, height - filter_size[0] + 1, strides[0]):
+ for j in range(0, width - filter_size[1] + 1, strides[1]):
+ receptive_field = input_matrix[i:i + filter_size[0], j:j + filter_size[1], :]
+ output_matrix[i // strides[0], j // strides[1], :] = np.sum(receptive_field * filter_matrix, axis=(0, 1, 2))
+
+ return output_matrix
+
+ def maxPooling(self, input_matrix, pool_size=(2, 2), strides_pooling=(2, 2)):
+ input_height, input_width, input_channels = input_matrix.shape
+ pool_height, pool_width = pool_size
+ stride_height, stride_width = strides_pooling
+
+ # Calculate output dimensions
+ pooled_height = (input_height - pool_height) // stride_height + 1
+ pooled_width = (input_width - pool_width) // stride_width + 1
+
+ # Initialize output
+ pooled_matrix = np.zeros((pooled_height, pooled_width, input_channels))
+
+ # Perform max pooling
+ for c in range(input_channels):
+ for i in range(0, input_height - pool_height + 1, stride_height):
+ for j in range(0, input_width - pool_width + 1, stride_width):
+ patch = input_matrix[i:i + pool_height, j:j + pool_width, c]
+ pooled_matrix[i // stride_height, j // stride_width, c] = np.max(patch)
+
+ return pooled_matrix
+
+ def flatten(self, input_matrix):
+ return input_matrix.flatten()
+
+ def dropout(self, input_matrix, dropout_rate=0.5):
+ assert 0 <= dropout_rate < 1, "Dropout rate must be in [0, 1)."
+ dropout_mask = np.random.binomial(1, 1 - dropout_rate, size=input_matrix.shape)
+ return input_matrix * dropout_mask
+```
+
+Run the below command to generate output with random input and filter matrices, depending on the given size.
+
+```python
+input_shape = (5, 5)
+channels = 1
+strides = (1, 1)
+padding = 'valid'
+filter_size = (3, 3)
+
+cnn_model = CNN()
+
+conv_output = cnn_model.convLayer(input_shape, channels, strides, padding, filter_size)
+print("\nConvolution Output:\n", conv_output[:, :, 0])
+
+pool_size = (2, 2)
+strides_pooling = (1, 1)
+
+maxPool_output = cnn_model.maxPooling(conv_output, pool_size, strides_pooling)
+print("\nMax Pooling Output:\n", maxPool_output[:, :, 0])
+
+flattened_output = cnn_model.flatten(maxPool_output)
+print("\nFlattened Output:\n", flattened_output)
+
+dropout_output = cnn_model.dropout(flattened_output, dropout_rate=0.3)
+print("\nDropout Output:\n", dropout_output)
+```
+
+Feel free to play around with the parameters!
diff --git a/contrib/machine-learning/kmeans.md b/contrib/machine-learning/kmeans.md
new file mode 100644
index 00000000..52db92e5
--- /dev/null
+++ b/contrib/machine-learning/kmeans.md
@@ -0,0 +1,92 @@
+# K-Means Clustering
+Unsupervised Learning Algorithm for Grouping Similar Data.
+
+## Introduction
+K-means clustering is a fundamental unsupervised machine learning algorithm that excels at grouping similar data points together. It's a popular choice due to its simplicity and efficiency in uncovering hidden patterns within unlabeled datasets.
+
+## Unsupervised Learning
+Unlike supervised learning algorithms that rely on labeled data for training, unsupervised algorithms, like K-means, operate solely on input data (without predefined categories). Their objective is to discover inherent structures or groupings within the data.
+
+## The K-Means Objective
+Organize similar data points into clusters to unveil underlying patterns. The main objective is to minimize total intra-cluster variance or the squared function.
+
+
+## Clusters and Centroids
+A cluster represents a collection of data points that share similar characteristics. K-means identifies a pre-determined number (k) of clusters within the dataset. Each cluster is represented by a centroid, which acts as its central point (imaginary or real).
+
+## Minimizing In-Cluster Variation
+The K-means algorithm strategically assigns each data point to a cluster such that the total variation within each cluster (measured by the sum of squared distances between points and their centroid) is minimized. In simpler terms, K-means strives to create clusters where data points are close to their respective centroids.
+
+## The Meaning Behind "K-Means"
+The "means" in K-means refers to the averaging process used to compute the centroid, essentially finding the center of each cluster.
+
+## K-Means Algorithm in Action
+
+The K-means algorithm follows an iterative approach to optimize cluster formation:
+
+1. **Initial Centroid Placement:** The process begins with randomly selecting k centroids to serve as initial reference points for each cluster.
+2. **Data Point Assignment:** Each data point is assigned to the closest centroid, effectively creating a preliminary clustering.
+3. **Centroid Repositioning:** Once data points are assigned, the centroids are recalculated by averaging the positions of the points within their respective clusters. These new centroids represent the refined centers of the clusters.
+4. **Iteration Until Convergence:** Steps 2 and 3 are repeated iteratively until a stopping criterion is met. This criterion can be either:
+ - **Centroid Stability:** No significant change occurs in the centroids' positions, indicating successful clustering.
+ - **Reaching Maximum Iterations:** A predefined number of iterations is completed.
+
+## Code
+Following is a simple implementation of K-Means.
+
+```python
+# Generate and Visualize Sample Data
+# import the necessary Libraries
+
+import numpy as np
+import matplotlib.pyplot as plt
+
+# Create data points for cluster 1 and cluster 2
+X = -2 * np.random.rand(100, 2)
+X1 = 1 + 2 * np.random.rand(50, 2)
+
+# Combine data points from both clusters
+X[50:100, :] = X1
+
+# Plot data points and display the plot
+plt.scatter(X[:, 0], X[:, 1], s=50, c='b')
+plt.show()
+
+# K-Means Model Creation and Training
+from sklearn.cluster import KMeans
+
+# Create KMeans object with 2 clusters
+kmeans = KMeans(n_clusters=2)
+kmeans.fit(X) # Train the model on the data
+
+# Visualize Data Points with Centroids
+centroids = kmeans.cluster_centers_ # Get centroids (cluster centers)
+
+plt.scatter(X[:, 0], X[:, 1], s=50, c='b') # Plot data points again
+plt.scatter(centroids[0, 0], centroids[0, 1], s=200, c='g', marker='s') # Plot centroid 1
+plt.scatter(centroids[1, 0], centroids[1, 1], s=200, c='r', marker='s') # Plot centroid 2
+plt.show() # Display the plot with centroids
+
+# Predict Cluster Label for New Data Point
+new_data = np.array([-3.0, -3.0])
+new_data_reshaped = new_data.reshape(1, -1)
+predicted_cluster = kmeans.predict(new_data_reshaped)
+print("Predicted cluster for new data:", predicted_cluster)
+```
+
+### Output:
+Before Implementing K-Means Clustering
+
+
+After Implementing K-Means Clustering
+
+
+Predicted cluster for new data: `[0]`
+
+## Conclusion
+**K-Means** can be applied to data that has a smaller number of dimensions, is numeric, and is continuous or can be used to find groups that have not been explicitly labeled in the data. As an example, it can be used for Document Classification, Delivery Store Optimization, or Customer Segmentation.
+
+## References
+
+- [Survey of Machine Learning and Data Mining Techniques used in Multimedia System](https://www.researchgate.net/publication/333457161_Survey_of_Machine_Learning_and_Data_Mining_Techniques_used_in_Multimedia_System?_tp=eyJjb250ZXh0Ijp7ImZpcnN0UGFnZSI6Il9kaXJlY3QiLCJwYWdlIjoiX2RpcmVjdCJ9fQ)
+- [A Clustering Approach for Outliers Detection in a Big Point-of-Sales Database](https://www.researchgate.net/publication/339267868_A_Clustering_Approach_for_Outliers_Detection_in_a_Big_Point-of-Sales_Database?_tp=eyJjb250ZXh0Ijp7ImZpcnN0UGFnZSI6Il9kaXJlY3QiLCJwYWdlIjoiX2RpcmVjdCJ9fQ)
diff --git a/contrib/machine-learning/knn.md b/contrib/machine-learning/knn.md
new file mode 100644
index 00000000..85578f3f
--- /dev/null
+++ b/contrib/machine-learning/knn.md
@@ -0,0 +1,122 @@
+# K-Nearest Neighbors (KNN) Machine Learning Algorithm in Python
+
+## Introduction
+K-Nearest Neighbors (KNN) is a simple, yet powerful, supervised machine learning algorithm used for both classification and regression tasks. It assumes that similar things exist in close proximity. In other words, similar data points are near to each other.
+
+## How KNN Works
+KNN works by finding the distances between a query and all the examples in the data, selecting the specified number of examples (K) closest to the query, then voting for the most frequent label (in classification) or averaging the labels (in regression).
+
+### Steps:
+1. **Choose the number K of neighbors**
+2. **Calculate the distance** between the query-instance and all the training samples
+3. **Sort the distances** and determine the nearest neighbors based on the K-th minimum distance
+4. **Gather the labels** of the nearest neighbors
+5. **Vote for the most frequent label** (in case of classification) or **average the labels** (in case of regression)
+
+## When to Use KNN
+### Advantages:
+- **Simple and easy to understand:** KNN is intuitive and easy to implement.
+- **No training phase:** KNN is a lazy learner, meaning there is no explicit training phase.
+- **Effective with a small dataset:** KNN performs well with a small number of input variables.
+
+### Disadvantages:
+- **Computationally expensive:** The algorithm becomes significantly slower as the number of examples and/or predictors/independent variables increase.
+- **Sensitive to irrelevant features:** All features contribute to the distance equally.
+- **Memory-intensive:** Storing all the training data can be costly.
+
+### Use Cases:
+- **Recommender Systems:** Suggest items based on similarity to user preferences.
+- **Image Recognition:** Classify images by comparing new images to the training set.
+- **Finance:** Predict credit risk or fraud detection based on historical data.
+
+## KNN in Python
+
+### Required Libraries
+To implement KNN, we need the following Python libraries:
+- `numpy`
+- `pandas`
+- `scikit-learn`
+- `matplotlib` (for visualization)
+
+### Installation
+```bash
+pip install numpy pandas scikit-learn matplotlib
+```
+
+### Example Code
+Let's implement a simple KNN classifier using the Iris dataset.
+
+#### Step 1: Import Libraries
+```python
+import numpy as np
+import pandas as pd
+from sklearn.model_selection import train_test_split
+from sklearn.neighbors import KNeighborsClassifier
+from sklearn.metrics import accuracy_score
+import matplotlib.pyplot as plt
+```
+
+#### Step 2: Load Dataset
+```python
+from sklearn.datasets import load_iris
+iris = load_iris()
+X = iris.data
+y = iris.target
+```
+
+#### Step 3: Split Dataset
+```python
+X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.3, random_state=42)
+```
+
+#### Step 4: Train KNN Model
+```python
+knn = KNeighborsClassifier(n_neighbors=3)
+knn.fit(X_train, y_train)
+```
+
+#### Step 5: Make Predictions
+```python
+y_pred = knn.predict(X_test)
+```
+
+#### Step 6: Evaluate the Model
+```python
+accuracy = accuracy_score(y_test, y_pred)
+print(f'Accuracy: {accuracy}')
+```
+
+### Visualization (Optional)
+```python
+# Plotting the decision boundary for visualization (for 2D data)
+h = .02 # step size in the mesh
+# Create color maps
+cmap_light = plt.cm.RdYlBu
+cmap_bold = plt.cm.RdYlBu
+
+# For simplicity, we take only the first two features of the dataset
+X_plot = X[:, :2]
+x_min, x_max = X_plot[:, 0].min() - 1, X_plot[:, 0].max() + 1
+y_min, y_max = X_plot[:, 1].min() - 1, y_plot[:, 1].max() + 1
+xx, yy = np.meshgrid(np.arange(x_min, x_max, h),
+ np.arange(y_min, y_max, h))
+
+Z = knn.predict(np.c_[xx.ravel(), yy.ravel()])
+Z = Z.reshape(xx.shape)
+plt.figure()
+plt.pcolormesh(xx, yy, Z, cmap=cmap_light)
+
+# Plot also the training points
+plt.scatter(X_plot[:, 0], X_plot[:, 1], c=y, edgecolor='k', cmap=cmap_bold)
+plt.xlim(xx.min(), xx.max())
+plt.ylim(yy.min(), yy.max())
+plt.title("3-Class classification (k = 3)")
+plt.show()
+```
+
+## Generalization and Considerations
+- **Choosing K:** The choice of K is critical. Smaller values of K can lead to noisy models, while larger values make the algorithm computationally expensive and might oversimplify the model.
+- **Feature Scaling:** Since KNN relies on distance calculations, features should be scaled (standardized or normalized) to ensure that all features contribute equally to the distance computation.
+- **Distance Metrics:** The choice of distance metric (Euclidean, Manhattan, etc.) can affect the performance of the algorithm.
+
+In conclusion, KNN is a versatile and easy-to-implement algorithm suitable for various classification and regression tasks, particularly when working with small datasets and well-defined features. However, careful consideration should be given to the choice of K, feature scaling, and distance metrics to optimize its performance.
diff --git a/contrib/machine-learning/naive-bayes.md b/contrib/machine-learning/naive-bayes.md
new file mode 100644
index 00000000..4bf0f04c
--- /dev/null
+++ b/contrib/machine-learning/naive-bayes.md
@@ -0,0 +1,328 @@
+# Naive Bayes
+
+## Introduction
+
+The Naive Bayes model uses probabilities to predict an outcome.It is a supervised machine learning technique, i.e. it reqires labelled data for training. It is used for classification and is based on the Bayes' Theorem. The basic assumption of this model is the independence among the features, i.e. a feature is unaffected by any other feture.
+
+## Bayes' Theorem
+
+Bayes' theorem is given by:
+
+$$
+P(a|b) = \frac{P(b|a)*P(a)}{P(b)}
+$$
+
+where:
+- $P(a|b)$ is the posterior probability, i.e. probability of 'a' given that 'b' is true,
+- $P(b|a)$ is the likelihood probability i.e. probability of 'b' given that 'a' is true,
+- $P(a)$ and $P(b)$ are the probabilities of 'a' and 'b' respectively, independent of each other.
+
+
+## Applications
+
+Naive Bayes classifier has numerous applications including :
+ 1. Text classification.
+ 2. Sentiment analysis.
+ 3. Spam filtering.
+ 4. Multiclass classification (eg. Weather prediction).
+ 5. Recommendation Systems.
+ 6. Healthcare sector.
+ 7. Document categorization.
+
+
+## Advantages
+
+ 1. Easy to implement.
+ 2. Useful even if training dataset is limited (where a decision tree would not be recommended).
+ 3. Supports multiclass classification which is not supported by some machine learning algorithms like SVM and logistic regression.
+ 4. Scalable, fast and efficient.
+
+## Disadvantages
+
+ 1. Assumes features to be independent, which may not be true in certain scenarios.
+ 2. Zero probability error.
+ 3. Sensitive to noise.
+
+## Zero Probability Error
+
+ Zero probability error is said to occur if in some case the number of occurances of an event given another event is zero.
+ To handle zero probability error, Laplace's correction is used by adding a small constant .
+
+**Example:**
+
+
+Given the data below, find whether tennis can be played if ( outlook=overcast, wind=weak ).
+
+**Data**
+
+---
+| SNo | Outlook (A) | Wind (B) | PlayTennis (R) |
+|-----|--------------|------------|-------------------|
+| 1 | Rain | Weak | No |
+| 2 | Rain | Strong | No |
+| 3 | Overcast | Weak | Yes |
+| 4 | Rain | Weak | Yes |
+| 5 | Overcast | Weak | Yes |
+| 6 | Rain | Strong | No |
+| 7 | Overcast | Strong | Yes |
+| 8 | Rain | Weak | No |
+| 9 | Overcast | Weak | Yes |
+| 10 | Rain | Weak | Yes |
+---
+
+- **Calculate prior probabilities**
+
+$$
+ P(Yes) = \frac{6}{10} = 0.6
+$$
+$$
+ P(No) = \frac{4}{10} = 0.4
+$$
+
+- **Calculate likelihoods**
+
+ 1.**Outlook (A):**
+
+ ---
+ | A\R | Yes | No |
+ |-----------|-------|-----|
+ | Rain | 2 | 4 |
+ | Overcast | 4 | 0 |
+ | Total | 6 | 4 |
+ ---
+
+- Rain:
+
+$$P(Rain|Yes) = \frac{2}{6}$$
+
+$$P(Rain|No) = \frac{4}{4}$$
+
+- Overcast:
+
+$$
+ P(Overcast|Yes) = \frac{4}{6}
+$$
+$$
+ P(Overcast|No) = \frac{0}{4}
+$$
+
+
+Here, we can see that P(Overcast|No) = 0
+This is a zero probability error!
+
+Since probability is 0, naive bayes model fails to predict.
+
+ **Applying Laplace's correction:**
+
+ In Laplace's correction, we scale the values for 1000 instances.
+ - **Calculate prior probabilities**
+
+ $$P(Yes) = \frac{600}{1002}$$
+
+ $$P(No) = \frac{402}{1002}$$
+
+- **Calculate likelihoods**
+
+ 1. **Outlook (A):**
+
+
+ ( Converted to 1000 instances )
+
+ We will add 1 instance each to the (PlayTennis|No) column {Laplace's correction}
+
+ ---
+ | A\R | Yes | No |
+ |-----------|-------|---------------|
+ | Rain | 200 | (400+1)=401 |
+ | Overcast | 400 | (0+1)=1 |
+ | Total | 600 | 402 |
+ ---
+
+ - **Rain:**
+
+ $$P(Rain|Yes) = \frac{200}{600}$$
+ $$P(Rain|No) = \frac{401}{402}$$
+
+ - **Overcast:**
+
+ $$P(Overcast|Yes) = \frac{400}{600}$$
+ $$P(Overcast|No) = \frac{1}{402}$$
+
+
+ 2. **Wind (B):**
+
+
+ ---
+ | B\R | Yes | No |
+ |-----------|---------|-------|
+ | Weak | 500 | 200 |
+ | Strong | 100 | 200 |
+ | Total | 600 | 400 |
+ ---
+
+ - **Weak:**
+
+ $$P(Weak|Yes) = \frac{500}{600}$$
+ $$P(Weak|No) = \frac{200}{400}$$
+
+ - **Strong:**
+
+ $$P(Strong|Yes) = \frac{100}{600}$$
+ $$P(Strong|No) = \frac{200}{400}$$
+
+ - **Calculting probabilities:**
+
+ $$P(PlayTennis|Yes) = P(Yes) * P(Overcast|Yes) * P(Weak|Yes)$$
+ $$= \frac{600}{1002} * \frac{400}{600} * \frac{500}{600}$$
+ $$= 0.3326$$
+
+ $$P(PlayTennis|No) = P(No) * P(Overcast|No) * P(Weak|No)$$
+ $$= \frac{402}{1002} * \frac{1}{402} * \frac{200}{400}$$
+ $$= 0.000499 = 0.0005$$
+
+
+Since ,
+$$P(PlayTennis|Yes) > P(PlayTennis|No)$$
+we can conclude that tennis can be played if outlook is overcast and wind is weak.
+
+
+# Types of Naive Bayes classifier
+
+
+## Guassian Naive Bayes
+
+ It is used when the dataset has **continuous data**. It assumes that the data is distributed normally (also known as guassian distribution).
+ A guassian distribution can be characterized by a bell-shaped curve.
+
+ **Continuous data features :** Features which can take any real values within a certain range. These features have an infinite number of possible values.They are generally measured, not counted.
+ eg. weight, height, temperature, etc.
+
+ **Code**
+
+ ```python
+
+#import libraries
+import pandas as pd
+from sklearn.model_selection import train_test_split
+from sklearn.naive_bayes import GaussianNB
+from sklearn import metrics
+from sklearn.metrics import confusion_matrix
+
+#read data
+d=pd.read_csv("data.csv")
+df=pd.DataFrame(d)
+
+X = df.iloc[:,1:7:1]
+y = df.iloc[:,7:8:1]
+
+# splitting X and y into training and testing sets
+X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.20, random_state=42)
+
+
+# training the model on training set
+obj = GaussianNB()
+obj.fit(X_train, y_train)
+
+#making predictions on the testing set
+y_pred = obj.predict(X_train)
+
+#comparing y_test and y_pred
+print("Gaussian Naive Bayes model accuracy:", metrics.accuracy_score(y_train, y_pred))
+print("Confusion matrix: \n",confusion_matrix(y_train,y_pred))
+
+ ```
+
+
+## Multinomial Naive Bayes
+
+ Appropriate when the features are categorical or countable. It models the likelihood of each feature as a multinomial distribution.
+ Multinomial distribution is used to find probabilities of each category, given multiple categories (eg. Text classification).
+
+ **Code**
+
+ ```python
+
+#import libraries
+import pandas as pd
+from sklearn.model_selection import train_test_split
+from sklearn.naive_bayes import MultinomialNB
+from sklearn import metrics
+from sklearn.metrics import confusion_matrix
+
+#read data
+d=pd.read_csv("data.csv")
+df=pd.DataFrame(d)
+
+X = df.iloc[:,1:7:1]
+y = df.iloc[:,7:8:1]
+
+# splitting X and y into training and testing sets
+X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.20, random_state=42)
+
+
+# training the model on training set
+obj = MultinomialNB()
+obj.fit(X_train, y_train)
+
+#making predictions on the testing set
+y_pred = obj.predict(X_train)
+
+#comparing y_test and y_pred
+print("Gaussian Naive Bayes model accuracy:", metrics.accuracy_score(y_train, y_pred))
+print("Confusion matrix: \n",confusion_matrix(y_train,y_pred))
+
+
+ ```
+
+## Bernoulli Naive Bayes
+
+ It is specifically designed for binary features (eg. Yes or No). It models the likelihood of each feature as a Bernoulli distribution.
+ Bernoulli distribution is used when there are only two possible outcomes (eg. success or failure of an event).
+
+ **Code**
+
+ ```python
+
+#import libraries
+import pandas as pd
+from sklearn.model_selection import train_test_split
+from sklearn.naive_bayes import BernoulliNB
+from sklearn import metrics
+from sklearn.metrics import confusion_matrix
+
+#read data
+d=pd.read_csv("data.csv")
+df=pd.DataFrame(d)
+
+X = df.iloc[:,1:7:1]
+y = df.iloc[:,7:8:1]
+
+# splitting X and y into training and testing sets
+X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.20, random_state=42)
+
+
+# training the model on training set
+obj = BernoulliNB()
+obj.fit(X_train, y_train)
+
+#making predictions on the testing set
+y_pred = obj.predict(X_train)
+
+#comparing y_test and y_pred
+print("Gaussian Naive Bayes model accuracy:", metrics.accuracy_score(y_train, y_pred))
+print("Confusion matrix: \n",confusion_matrix(y_train,y_pred))
+
+ ```
+
+
+## Evaluation
+
+ 1. Confusion matrix.
+ 2. Accuracy.
+ 3. ROC curve.
+
+
+## Conclusion
+
+ We can conclude that naive bayes may limit in some cases due to the assumption that the features are independent of each other but still reliable in many cases. Naive Bayes is an efficient classifier and works even on small datasets.
+
diff --git a/contrib/machine-learning/neural-network-regression.md b/contrib/machine-learning/neural-network-regression.md
new file mode 100644
index 00000000..aa16bc6c
--- /dev/null
+++ b/contrib/machine-learning/neural-network-regression.md
@@ -0,0 +1,84 @@
+# Neural Network Regression in Python using Scikit-learn
+
+## Overview
+
+Neural Network Regression is used to predict continuous values based on input features. Scikit-learn provides an easy-to-use interface for implementing neural network models, specifically through the `MLPRegressor` class, which stands for Multi-Layer Perceptron Regressor.
+
+## When to Use Neural Network Regression
+
+### Suitable Scenarios
+
+1. **Complex Relationships**: Ideal when the relationship between features and the target variable is complex and non-linear.
+2. **Sufficient Data**: Works well with large datasets that can support training deep learning models.
+3. **Feature Extraction**: Useful in cases where the neural network's feature extraction capabilities can be leveraged, such as with image or text data.
+
+### Unsuitable Scenarios
+
+1. **Small Datasets**: Less effective with small datasets due to overfitting and inability to learn complex patterns.
+2. **Low-latency Predictions**: Might not be suitable for real-time applications with strict latency requirements.
+3. **Interpretability**: Not ideal when model interpretability is crucial, as neural networks are often seen as "black-box" models.
+
+## Implementing Neural Network Regression in Python with Scikit-learn
+
+### Step-by-Step Implementation
+
+1. **Import Libraries**
+
+```python
+import numpy as np
+import pandas as pd
+from sklearn.model_selection import train_test_split
+from sklearn.preprocessing import StandardScaler
+from sklearn.neural_network import MLPRegressor
+from sklearn.metrics import mean_absolute_error
+```
+
+2. **Load and Prepare Data**
+
+For illustration, let's use a synthetic dataset.
+
+```python
+# Generate synthetic data
+np.random.seed(42)
+X = np.random.rand(1000, 3)
+y = X[:, 0] * 3 + X[:, 1] * -2 + X[:, 2] * 0.5 + np.random.randn(1000) * 0.1
+
+# Split the data
+X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)
+
+# Standardize the data
+scaler = StandardScaler()
+X_train = scaler.fit_transform(X_train)
+X_test = scaler.transform(X_test)
+```
+
+3. **Build and Train the Neural Network Model**
+
+```python
+# Create the MLPRegressor model
+mlp = MLPRegressor(hidden_layer_sizes=(64, 64), activation='relu', solver='adam', max_iter=500, random_state=42)
+
+# Train the model
+mlp.fit(X_train, y_train)
+```
+
+4. **Evaluate the Model**
+
+```python
+# Make predictions
+y_pred = mlp.predict(X_test)
+
+# Calculate the Mean Absolute Error
+mae = mean_absolute_error(y_test, y_pred)
+print(f"Test Mean Absolute Error: {mae}")
+```
+
+### Explanation
+
+- **Data Generation and Preparation**: Synthetic data is created, split into training and test sets, and standardized to improve the efficiency of the neural network training process.
+- **Model Construction and Training**: An `MLPRegressor` is created with two hidden layers, each containing 64 neurons and ReLU activation functions. The model is trained using the Adam optimizer for a maximum of 500 iterations.
+- **Evaluation**: The model's performance is evaluated on the test set using Mean Absolute Error (MAE) as the performance metric.
+
+## Conclusion
+
+Neural Network Regression with Scikit-learn's `MLPRegressor` is a powerful method for predicting continuous values in complex, non-linear scenarios. However, it's essential to ensure that you have enough data to train the model effectively and consider the computational resources required. Simpler models may be more appropriate for small datasets or when model interpretability is necessary. By following the steps outlined, you can build, train, and evaluate a neural network for regression tasks in Python using Scikit-learn.
diff --git a/contrib/machine-learning/polynomial-regression.md b/contrib/machine-learning/polynomial-regression.md
new file mode 100644
index 00000000..d00ede3b
--- /dev/null
+++ b/contrib/machine-learning/polynomial-regression.md
@@ -0,0 +1,102 @@
+# Polynomial Regression
+
+Polynomial Regression is a form of regression analysis in which the relationship between the independent variable $x$ and the dependent variable $y$ is modeled as an $nth$ degree polynomial. This guide provides an overview of polynomial regression, including its fundamental concepts, assumptions, and how to implement it using Python.
+
+## Introduction
+
+Polynomial Regression is used when the data shows a non-linear relationship between the independent variable $x$ and the dependent variable $y$ is modeled as an $nth$ degree polynomial. It extends the simple linear regression model by considering polynomial terms of the independent variable, allowing for a more flexible fit to the data.
+
+## Concepts
+
+### Polynomial Equation
+
+The polynomial regression model is based on the following polynomial equation:
+
+$$
+\[ y = \beta_0 + \beta_1 x + \beta_2 x^2 + \beta_3 x^3 + \cdots + \beta_n x^n + \epsilon \]
+$$
+
+Where:
+- $y$ is the dependent variable.
+- $x$ is the independent variable.
+- $\beta_0, \beta_1, \ldots, \beta_n$ are the coefficients of the polynomial.
+- $\epsilon$ is the error term.
+
+### Degree of Polynomial
+
+The degree of the polynomial (n) determines the flexibility of the model. A higher degree allows the model to fit more complex, non-linear relationships, but it also increases the risk of overfitting.
+
+### Overfitting and Underfitting
+
+- **Overfitting**: When the model fits the noise in the training data too closely, resulting in poor generalization to new data.
+- **Underfitting**: When the model is too simple to capture the underlying pattern in the data.
+
+## Assumptions
+
+1. **Independence**: Observations are independent of each other.
+2. **Homoscedasticity**: The variance of the residuals (errors) is constant across all levels of the independent variable.
+3. **Normality**: The residuals of the model are normally distributed.
+4. **No Multicollinearity**: The predictor variables are not highly correlated with each other.
+
+## Implementation
+
+### Using Scikit-learn
+
+Scikit-learn is a popular machine learning library in Python that provides tools for polynomial regression.
+
+### Code Example
+
+```python
+import numpy as np
+import pandas as pd
+import matplotlib.pyplot as plt
+from sklearn.preprocessing import PolynomialFeatures
+from sklearn.linear_model import LinearRegression
+from sklearn.metrics import mean_squared_error, r2_score
+
+# Load dataset
+data = pd.read_csv('path/to/your/dataset.csv')
+
+# Define features and target variable
+X = data[['feature']]
+y = data['target']
+
+# Transform features to polynomial features
+poly = PolynomialFeatures(degree=3)
+X_poly = poly.fit_transform(X)
+
+# Initialize and train polynomial regression model
+model = LinearRegression()
+model.fit(X_poly, y)
+
+# Make predictions
+y_pred = model.predict(X_poly)
+
+# Evaluate the model
+mse = mean_squared_error(y, y_pred)
+r2 = r2_score(y, y_pred)
+print("Mean Squared Error:", mse)
+print("R^2 Score:", r2)
+
+# Visualize the results
+plt.scatter(X, y, color='blue')
+plt.plot(X, y_pred, color='red')
+plt.xlabel('Feature')
+plt.ylabel('Target')
+plt.title('Polynomial Regression')
+plt.show()
+```
+
+## Evaluation Metrics
+
+- **Mean Squared Error (MSE)**: The average of the squared differences between actual and predicted values.
+- **R-squared (R²) Score**: A statistical measure that represents the proportion of the variance for the dependent variable that is explained by the independent variables in the model.
+
+## Conclusion
+
+Polynomial Regression is a powerful tool for modeling non-linear relationships between variables. It is important to choose the degree of the polynomial carefully to balance between underfitting and overfitting. Understanding and properly evaluating the model using appropriate metrics ensures its effectiveness.
+
+## References
+
+- [Scikit-learn Documentation](https://scikit-learn.org/stable/modules/linear_model.html#polynomial-regression)
+- [Wikipedia: Polynomial Regression](https://en.wikipedia.org/wiki/Polynomial_reg)
diff --git a/contrib/machine-learning/pytorch-fundamentals.md b/contrib/machine-learning/pytorch-fundamentals.md
new file mode 100644
index 00000000..b244ec1f
--- /dev/null
+++ b/contrib/machine-learning/pytorch-fundamentals.md
@@ -0,0 +1,469 @@
+# PyTorch Fundamentals
+
+
+```python
+# Import pytorch in our codespace
+import torch
+print(torch.__version__)
+```
+
+#### Output
+```
+2.3.0+cu121
+```
+
+
+2.3.0 is the pytorch version and 121 is the cuda version
+
+Now you have already seen how to create a tensor in pytorch. In this notebook i am going to show you the operations which can be applied on a tensor with a quick previous revision.
+
+### 1. Creating tensors
+
+Scalar tensor ( a zero dimension tensor)
+
+```python
+scalar = torch.tensor(7)
+print(scalar)
+```
+
+#### Output
+```
+tensor(7)
+```
+
+Check the dimension of the above tensor
+
+```python
+print(scalar.ndim)
+```
+
+#### Output
+```
+0
+```
+
+To retrieve the number from the tensor we use `item()`
+
+```python
+print(scalar.item())
+```
+
+#### Output
+```
+7
+```
+
+Vector (It is a single dimension tensor but contain many numbers)
+
+```python
+vector = torch.tensor([1,2])
+print(vector)
+```
+
+#### Output
+```
+tensor([1, 2])
+```
+
+Check the dimensions
+
+```python
+print(vector.ndim)
+```
+
+#### Output
+```
+1
+```
+
+Check the shape of the vector
+
+```python
+print(vector.shape)
+```
+
+#### Output
+```
+torch.Size([2])
+```
+
+
+The above returns torch.Size([2]) which means our vector has a shape of [2]. This is because of the two elements we placed inside the square brackets ([1,2])
+
+Note:
+I'll let you in on a trick.
+
+You can tell the number of dimensions a tensor in PyTorch has by the number of square brackets on the outside ([) and you only need to count one side.
+
+
+```python
+# Let's create a matrix
+MATRIX = torch.tensor([[1,2],
+ [4,5]])
+print(MATRIX)
+```
+
+#### Output
+```
+tensor([[1, 2],
+ [4, 5]])
+```
+
+There are two brackets so it must be 2 dimensions , lets check
+
+
+```python
+print(MATRIX.ndim)
+```
+
+#### Output
+```
+2
+```
+
+
+```python
+# Shape
+print(MATRIX.shape)
+```
+
+#### Output
+```
+torch.Size([2, 2])
+```
+
+It means MATRIX has 2 rows and 2 columns.
+
+Let's create a TENSOR
+
+```python
+TENSOR = torch.tensor([[[1,2,3],
+ [4,5,6],
+ [7,8,9]]])
+print(TENSOR)
+```
+
+#### Output
+```
+tensor([[[1, 2, 3],
+ [4, 5, 6],
+ [7, 8, 9]]])
+```
+
+Let's check the dimensions
+```python
+print(TENSOR.ndim)
+```
+
+#### Output
+```
+3
+```
+
+shape
+```python
+print(TENSOR.shape)
+```
+
+#### Output
+```
+torch.Size([1, 3, 3])
+```
+
+The dimensions go outer to inner.
+
+That means there's 1 dimension of 3 by 3.
+
+##### Let's summarise
+
+* scalar -> a single number having 0 dimension.
+* vector -> have many numbers but having 1 dimension.
+* matrix -> a array of numbers having 2 dimensions.
+* tensor -> a array of numbers having n dimensions.
+
+### Random Tensors
+
+We can create them using `torch.rand()` and passing in the `size` parameter.
+
+Creating a random tensor of size (3,4)
+```python
+rand_tensor = torch.rand(size = (3,4))
+print(rand_tensor)
+```
+
+#### Output
+```
+tensor([[0.7462, 0.4950, 0.7851, 0.8277],
+ [0.6112, 0.5159, 0.1728, 0.6847],
+ [0.4472, 0.1612, 0.6481, 0.3236]])
+```
+
+Check the dimensions
+
+```python
+print(rand_tensor.ndim)
+```
+
+#### Output
+```
+2
+```
+
+Shape
+```python
+print(rand_tensor.shape)
+```
+
+#### Output
+```
+torch.Size([3, 4])
+```
+
+Datatype
+```python
+print(rand_tensor.dtype)
+```
+
+#### Output
+```
+torch.float32
+```
+
+### Zeros and ones
+
+Here we will create a tensor of any shape filled with zeros and ones
+
+
+```python
+# Create a tensor of all zeros
+zeros = torch.zeros(size = (3,4))
+print(zeros)
+```
+
+#### Output
+```
+tensor([[0., 0., 0., 0.],
+ [0., 0., 0., 0.],
+ [0., 0., 0., 0.]])
+```
+
+Create a tensor of ones
+```python
+ones = torch.ones(size = (3,4))
+print(ones)
+```
+
+#### Output
+```
+tensor([[1., 1., 1., 1.],
+ [1., 1., 1., 1.],
+ [1., 1., 1., 1.]])
+```
+
+### Create a tensor having range of numbers
+
+You can use `torch.arange(start, end, step)` to do so.
+
+Where:
+
+* start = start of range (e.g. 0)
+* end = end of range (e.g. 10)
+* step = how many steps in between each value (e.g. 1)
+
+> Note: In Python, you can use range() to create a range. However in PyTorch, torch.range() is deprecated show error, show use `torch.arange()`
+
+
+```python
+zero_to_ten = torch.arange(start = 0,
+ end = 10,
+ step = 1)
+print(zero_to_ten)
+```
+
+#### Output
+```
+tensor([0, 1, 2, 3, 4, 5, 6, 7, 8, 9])
+```
+
+# 2. Manipulating tensors (tensor operations)
+
+The operations are :
+
+* Addition
+* Substraction
+* Multiplication (element-wise)
+* Division
+* Matrix multiplication
+
+### 1. Addition
+
+
+```python
+tensor = torch.tensor([1,2,3])
+print(tensor+10)
+```
+
+#### Output
+```
+tensor([11, 12, 13])
+```
+
+We have add 10 to each tensor element.
+
+
+```python
+tensor1 = torch.tensor([4,5,6])
+print(tensor+tensor1)
+```
+
+#### Output
+```
+tensor([5, 7, 9])
+```
+
+We have added two tensors , remember that addition takes place element wise.
+
+### 2. Subtraction
+
+
+```python
+print(tensor-8)
+```
+
+#### Output
+```
+tensor([-7, -6, -5])
+```
+
+We've subtracted 8 from the above tensor.
+
+
+```python
+print(tensor-tensor1)
+```
+
+#### Output
+```
+tensor([-3, -3, -3])
+```
+
+### 3. Multiplication
+
+
+```python
+# Multiply the tensor with 10 (element wise)
+print(tensor*10)
+```
+
+#### Output
+```
+tensor([10, 20, 30])
+```
+
+Each element of tensor gets multiplied by 10.
+
+Note:
+
+PyTorch also has a bunch of built-in functions like `torch.mul()` (short for multiplication) and `torch.add()` to perform basic operations.
+
+
+```python
+# let's see them
+print(torch.add(tensor,10))
+```
+
+#### Output
+```
+tensor([11, 12, 13])
+```
+
+
+```python
+print(torch.mul(tensor,10))
+```
+
+#### Output
+```
+tensor([10, 20, 30])
+```
+
+### Matrix multiplication (is all you need)
+One of the most common operations in machine learning and deep learning algorithms (like neural networks) is matrix multiplication.
+
+PyTorch implements matrix multiplication functionality in the `torch.matmul()` method.
+
+The main two rules for matrix multiplication to remember are:
+
+The inner dimensions must match:
+* (3, 2) @ (3, 2) won't work
+* (2, 3) @ (3, 2) will work
+* (3, 2) @ (2, 3) will work
+The resulting matrix has the shape of the outer dimensions:
+* (2, 3) @ (3, 2) -> (2, 2)
+* (3, 2) @ (2, 3) -> (3, 3)
+
+
+Note: "@" in Python is the symbol for matrix multiplication.
+
+
+```python
+# let's perform the matrix multiplication
+tensor1 = torch.tensor([[[1,2,3],
+ [4,5,6],
+ [7,8,9]]])
+tensor2 = torch.tensor([[[1,1,1],
+ [2,2,2],
+ [3,3,3]]])
+
+print(tensor1) , print(tensor2)
+
+```
+
+#### Output
+```
+tensor([[[1, 2, 3],
+ [4, 5, 6],
+ [7, 8, 9]]])
+tensor([[[1, 1, 1],
+ [2, 2, 2],
+ [3, 3, 3]]])
+```
+
+Let's check the shape
+```python
+print(tensor1.shape) , print(tensor2.shape)
+```
+
+#### Output
+```
+torch.Size([1, 3, 3])
+torch.Size([1, 3, 3])
+```
+
+Matrix multiplication
+```python
+print(torch.matmul(tensor1, tensor2))
+```
+
+#### Output
+```
+tensor([[[14, 14, 14],
+ [32, 32, 32],
+ [50, 50, 50]]])
+```
+
+Can also use the "@" symbol for matrix multiplication, though not recommended
+```python
+print(tensor1 @ tensor2)
+```
+
+#### Output
+```
+tensor([[[14, 14, 14],
+ [32, 32, 32],
+ [50, 50, 50]]])
+```
+
+Note:
+
+If shape is not perfect you can transpose the tensor and perform the matrix multiplication.
diff --git a/contrib/machine-learning/random-forest.md b/contrib/machine-learning/random-forest.md
new file mode 100644
index 00000000..feaaa7a7
--- /dev/null
+++ b/contrib/machine-learning/random-forest.md
@@ -0,0 +1,171 @@
+# Random Forest
+
+Random Forest is a versatile machine learning algorithm capable of performing both regression and classification tasks. It is an ensemble method that operates by constructing a multitude of decision trees during training and outputting the average prediction of the individual trees (for regression) or the mode of the classes (for classification).
+
+## Introduction
+Random Forest is an ensemble learning method used for classification and regression tasks. It is built from multiple decision trees and combines their outputs to improve the model's accuracy and control over-fitting.
+
+## How Random Forest Works
+### 1. Bootstrap Sampling:
+* Random subsets of the training dataset are created with replacement. Each subset is used to train an individual tree.
+### 2. Decision Trees:
+* Multiple decision trees are trained on these subsets.
+### 3. Feature Selection:
+* At each split in the decision tree, a random selection of features is chosen. This randomness helps create diverse trees.
+### 4. Voting/Averaging:
+For classification, the mode of the classes predicted by individual trees is taken (majority vote).
+For regression, the average of the outputs of the individual trees is taken.
+### Detailed Working Mechanism
+#### Step 1: Bootstrap Sampling:
+ Each tree is trained on a random sample of the original data, drawn with replacement (bootstrap sample). This means some data points may appear multiple times in a sample while others may not appear at all.
+#### Step 2: Tree Construction:
+ Each node in the tree is split using the best split among a random subset of the features. This process adds an additional layer of randomness, contributing to the robustness of the model.
+#### Step 3: Aggregation:
+ For classification tasks, the final prediction is based on the majority vote from all the trees. For regression tasks, the final prediction is the average of all the tree predictions.
+### Advantages and Disadvantages
+#### Advantages
+* Robustness: Reduces overfitting and generalizes well due to the law of large numbers.
+* Accuracy: Often provides high accuracy because of the ensemble method.
+* Versatility: Can be used for both classification and regression tasks.
+* Handles Missing Values: Can handle missing data better than many other algorithms.
+* Feature Importance: Provides estimates of feature importance, which can be valuable for understanding the model.
+#### Disadvantages
+* Complexity: More complex than individual decision trees, making interpretation difficult.
+* Computational Cost: Requires more computational resources due to multiple trees.
+* Training Time: Can be slow to train compared to simpler models, especially with large datasets.
+### Hyperparameters
+#### Key Hyperparameters
+* n_estimators: The number of trees in the forest.
+* max_features: The number of features to consider when looking for the best split.
+* max_depth: The maximum depth of the tree.
+* min_samples_split: The minimum number of samples required to split an internal node.
+* min_samples_leaf: The minimum number of samples required to be at a leaf node.
+* bootstrap: Whether bootstrap samples are used when building trees. If False, the whole dataset is used to build each tree.
+##### Tuning Hyperparameters
+Hyperparameter tuning can significantly improve the performance of a Random Forest model. Common techniques include Grid Search and Random Search.
+
+### Code Examples
+#### Classification Example
+Below is a simple example of using Random Forest for a classification task with the Iris dataset.
+
+```python
+import numpy as np
+import pandas as pd
+from sklearn.datasets import load_iris
+from sklearn.ensemble import RandomForestClassifier
+from sklearn.model_selection import train_test_split
+from sklearn.metrics import accuracy_score, classification_report
+
+
+# Load dataset
+iris = load_iris()
+X, y = iris.data, iris.target
+
+# Split dataset
+X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.3, random_state=42)
+
+# Initialize Random Forest model
+clf = RandomForestClassifier(n_estimators=100, random_state=42)
+
+# Train the model
+clf.fit(X_train, y_train)
+
+# Make predictions
+y_pred = clf.predict(X_test)
+
+# Evaluate the model
+accuracy = accuracy_score(y_test, y_pred)
+print(f"Accuracy: {accuracy * 100:.2f}%")
+print("Classification Report:\n", classification_report(y_test, y_pred))
+
+```
+
+#### Feature Importance
+Random Forest provides a way to measure the importance of each feature in making predictions.
+
+
+```python
+import matplotlib.pyplot as plt
+
+# Get feature importances
+importances = clf.feature_importances_
+indices = np.argsort(importances)[::-1]
+
+# Print feature ranking
+print("Feature ranking:")
+for f in range(X.shape[1]):
+ print(f"{f + 1}. Feature {indices[f]} ({importances[indices[f]]})")
+
+# Plot the feature importances
+plt.figure()
+plt.title("Feature importances")
+plt.bar(range(X.shape[1]), importances[indices], align='center')
+plt.xticks(range(X.shape[1]), indices)
+plt.xlim([-1, X.shape[1]])
+plt.show()
+```
+#### Hyperparameter Tuning
+Using Grid Search for hyperparameter tuning.
+
+```python
+from sklearn.model_selection import GridSearchCV
+
+# Define the parameter grid
+param_grid = {
+ 'n_estimators': [100, 200, 300],
+ 'max_features': ['auto', 'sqrt', 'log2'],
+ 'max_depth': [4, 6, 8, 10, 12],
+ 'criterion': ['gini', 'entropy']
+}
+
+# Initialize the Grid Search model
+grid_search = GridSearchCV(estimator=clf, param_grid=param_grid, cv=3, n_jobs=-1, verbose=2)
+
+# Fit the model
+grid_search.fit(X_train, y_train)
+
+# Print the best parameters
+print("Best parameters found: ", grid_search.best_params_)
+```
+#### Regression Example
+Below is a simple example of using Random Forest for a regression task with the Boston housing dataset.
+
+```python
+import numpy as np
+import pandas as pd
+from sklearn.datasets import load_boston
+from sklearn.ensemble import RandomForestRegressor
+from sklearn.model_selection import train_test_split
+from sklearn.metrics import mean_squared_error, r2_score
+
+# Load dataset
+boston = load_boston()
+X, y = boston.data, boston.target
+
+# Split dataset
+X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.3, random_state=42)
+
+# Initialize Random Forest model
+regr = RandomForestRegressor(n_estimators=100, random_state=42)
+
+# Train the model
+regr.fit(X_train, y_train)
+
+# Make predictions
+y_pred = regr.predict(X_test)
+
+# Evaluate the model
+mse = mean_squared_error(y_test, y_pred)
+r2 = r2_score(y_test, y_pred)
+print(f"Mean Squared Error: {mse:.2f}")
+print(f"R^2 Score: {r2:.2f}")
+```
+## Conclusion
+Random Forest is a powerful and flexible machine learning algorithm that can handle both classification and regression tasks. Its ability to create an ensemble of decision trees leads to robust and accurate models. However, it is important to be mindful of the computational cost associated with training multiple trees.
+
+## References
+Scikit-learn Random Forest Documentation
+Wikipedia: Random Forest
+Machine Learning Mastery: Introduction to Random Forest
+Kaggle: Random Forest Guide
+Towards Data Science: Understanding Random Forests
diff --git a/contrib/machine-learning/Regression.md b/contrib/machine-learning/regression.md
similarity index 100%
rename from contrib/machine-learning/Regression.md
rename to contrib/machine-learning/regression.md
diff --git a/contrib/machine-learning/reinforcement-learning.md b/contrib/machine-learning/reinforcement-learning.md
new file mode 100644
index 00000000..c5529fc9
--- /dev/null
+++ b/contrib/machine-learning/reinforcement-learning.md
@@ -0,0 +1,233 @@
+# Reinforcement Learning: A Comprehensive Guide
+
+Reinforcement Learning (RL) is a field of Machine Learing which focuses on goal-directed learning from interaction with the environment. In RL, an agent learns to make decisions by performing actions in an environment to maximize cumulative numerical reward signal. This README aims to provide a thorough understanding of RL, covering key concepts, algorithms, applications, and resources.
+
+## What is Reinforcement Learning?
+
+Reinforcement learning involves determining the best actions to take in various situations to maximize a numerical reward signal. Instead of being instructed on which actions to take, the learner must explore and identify the actions that lead to the highest rewards through trial and error. After each action performed in its environment, a trainer may give feedback in the form of rewards or penalties to indicate the desirability of the resulting state. Unlike supervised learning, reinforcement learning does not depend on labeled data but instead learns from the outcomes of its actions.
+
+## Key Concepts and Terminology
+
+### Agent
+Agent is a system or entity that learns to make decisions by interacting with an environment. The agent improves its performance by trial and error, receiving feedback from the environment in the form of rewards or punishments.
+
+### Environment
+Environment is the setting or world in which the agent operates and interacts with. It provides the agent with states and feedback based on the agent's actions.
+
+### State
+State represents the current situation of the environment, encapsulating all the relevant information needed for decision-making.
+
+### Action
+Action represents a move that can be taken by the agent, which would affect the state of the environment. The set of all possible actions is called the action space.
+
+### Reward
+Reward is the feedback from the environment in response to the agent’s action, thereby defining what are good and bad actions. Agent aims to maximize the total reward over time.
+
+### Policy
+Policy is a strategy used by the agent to determine its actions based on the current state. In some cases the policy may be a simple function or lookup table, whereas in others it may involve extensive computation such as a search process.
+
+### Value Function
+The value function of a state is the expected total amount of reward an agent can expect to accumulate over the future, starting from that state. There are two main types of value functions:
+ - **State Value Function (V)**: The expected reward starting from a state and following a certain policy thereafter.
+ - **Action Value Function (Q)**: The expected reward starting from a state, taking a specific action, and following a certain policy thereafter.
+
+### Model
+Model mimics the behavior of the environment, or more generally, that allows inferences to be made about how the environment will behave.
+
+### Exploration vs. Exploitation
+To accumulate substantial rewards, a reinforcement learning agent needs to favor actions that have previously yielded high rewards. However, to identify these effective actions, the agent must also attempt actions it hasn't tried before. This means the agent must *exploit* its past experiences to gain rewards, while also *exploring* new actions to improve its future decision-making.
+
+## Types of Reinforcement Learning
+
+### Model-Based vs Model-Free
+
+**Model-Based Reinforcement Learning:** Model-based methods involve creating a model of the environment to predict future states and rewards, allowing the agent to plan its actions by simulating various scenarios. These methods often involve two main components:
+
+**Model-Free Reinforcement Learning:** Model-free methods do not explicitly learn a model of the environment. Instead, they learn a policy or value function directly from the interactions with the environment. These methods can be further divided into two categories: value-based and policy-based methods.
+
+### Value-Based Methods:
+Value-based methods focus on estimating the value function, and the policy is indirectly derived from the value function.
+
+### Policy-Based Methods:
+Policy-based methods directly optimize the policy by maximizing the expected cumulative rewardto find the optimal parameters.
+
+### Actor-Critic Methods:
+Actor-Critic methods combine the strengths of both value-based and policy-based methods. Actor learns the policy that maps states to actions and Critic learns the value function that evaluates the action chosen by the actor.
+
+## Important Algorithms
+
+### Q-Learning
+Q-Learning is a model-free algorithm used in reinforcement learning to learn the value of an action in a particular state. It aims to find the optimal policy by iteratively updating the Q-values, which represent the expected cumulative reward of taking a particular action in a given state and following the optimal policy thereafter.
+
+#### Algorithm:
+1. Initialize Q-values arbitrarily for all state-action pairs.
+2. Repeat for each episode:
+ - Choose an action using an exploration strategy (e.g., epsilon-greedy).
+ - Take the action, observe the reward and the next state.
+ - Update the Q-value of the current state-action pair using the Bellman equation:
+ $$Q(s, a) \leftarrow Q(s, a) + \alpha \left( r + \gamma \max_{a'} Q(s', a') - Q(s, a) \right)$$
+
+ where:
+ - $Q(s, a)$ is the Q-value of state $s$ and action $a$.
+ - $r$ is the observed reward.
+ - $s'$ is the next state.
+ - $\alpha$ is the learning rate.
+ - $\gamma$ is the discount factor.
+3. Until convergence or a maximum number of episodes.
+
+### SARSA
+SARSA (State-Action-Reward-State-Action) is an on-policy temporal difference algorithm used for learning the Q-function. Unlike Q-learning, SARSA directly updates the Q-values based on the current policy.
+
+#### Algorithm:
+1. Initialize Q-values arbitrarily for all state-action pairs.
+2. Repeat for each episode:
+ - Initialize the environment state $s$.
+ - Choose an action $a$ using the current policy (e.g., epsilon-greedy).
+ - Repeat for each timestep:
+ - Take action $a$, observe the reward $r$ and the next state $s'$.
+ - Choose the next action $a'$ using the current policy.
+ - Update the Q-value of the current state-action pair using the SARSA update rule:
+ $$Q(s, a) \leftarrow Q(s, a) + \alpha \left( r + \gamma Q(s', a') - Q(s, a) \right)$$
+3. Until convergence or a maximum number of episodes.
+
+### REINFORCE Algorithm:
+REINFORCE (Monte Carlo policy gradient) is a simple policy gradient method that updates the policy parameters in the direction of the gradient of expected rewards.
+
+### Proximal Policy Optimization (PPO):
+PPO is an advanced policy gradient method that improves stability by limiting the policy updates within a certain trust region.
+
+### A2C/A3C:
+Advantage Actor-Critic (A2C) and Asynchronous Advantage Actor-Critic (A3C) are variants of actor-critic methods that utilize multiple parallel agents to improve sample efficiency.
+
+## Mathematical Background
+
+### Markov Decision Processes (MDPs)
+A Markov Decision Process (MDP) is a mathematical framework used to model decision-making problems. It consists of states, actions, rewards and transition probabilities.
+
+### Bellman Equations
+Bellman equations are fundamental recursive equations in dynamic programming and reinforcement learning. They express the value of a decision at one point in time in terms of the expected value of the subsequent decisions.
+
+## Applications of Reinforcement Learning
+
+### Gaming
+Reinforcement learning is extensively used in gaming for developing AI agents capable of playing complex games like AlphaGo, Chess, and video games. RL algorithms enable these agents to learn optimal strategies by interacting with the game environment and receiving feedback in the form of rewards.
+
+### Robotics
+In robotics, reinforcement learning is employed to teach robots various tasks such as navigation, manipulation, and control. RL algorithms allow robots to learn from their interactions with the environment, enabling them to adapt and improve their behavior over time without explicit programming.
+
+### Finance
+Reinforcement learning plays a crucial role in finance, particularly in algorithmic trading and portfolio management. RL algorithms are utilized to optimize trading strategies, automate decision-making processes, and manage investment portfolios dynamically based on changing market conditions and objectives.
+
+### Healthcare
+In healthcare, reinforcement learning is utilized for various applications such as personalized treatment, drug discovery, and optimizing healthcare operations. RL algorithms can assist in developing personalized treatment plans for patients, identifying effective drug candidates, and optimizing resource allocation in hospitals to improve patient care and outcomes.
+
+## Tools and Libraries
+- **OpenAI Gym:** A toolkit for developing and comparing RL algorithms.
+- **TensorFlow/TF-Agents:** A library for RL in TensorFlow.
+- **PyTorch:** Popular machine learning library with RL capabilities.
+- **Stable Baselines3:** A set of reliable implementations of RL algorithms in PyTorch.
+
+## How to Start with Reinforcement Learning
+
+### Prerequisites
+- Basic knowledge of machine learning and neural networks.
+- Proficiency in Python.
+
+### Beginner Project
+The provided Python code implements the Q-learning algorithm for a basic grid world environment. It defines the grid world, actions, and parameters such as discount factor and learning rate. The algorithm iteratively learns the optimal action-value function (Q-values) by updating them based on rewards obtained from actions taken in each state. Finally, the learned Q-values are printed for each state-action pair.
+
+```python
+import numpy as np
+
+# Define the grid world environment
+# 'S' represents the start state
+# 'G' represents the goal state
+# 'H' represents the hole (negative reward)
+# '.' represents empty cells (neutral reward)
+# 'W' represents walls (impassable)
+grid_world = np.array([
+ ['S', '.', '.', '.', '.'],
+ ['.', 'W', '.', 'H', '.'],
+ ['.', '.', '.', 'W', '.'],
+ ['.', 'W', '.', '.', 'G']
+])
+
+# Define the actions (up, down, left, right)
+actions = ['UP', 'DOWN', 'LEFT', 'RIGHT']
+
+# Define parameters
+gamma = 0.9 # discount factor
+alpha = 0.1 # learning rate
+epsilon = 0.1 # exploration rate
+
+# Initialize Q-values
+num_rows, num_cols = grid_world.shape
+num_actions = len(actions)
+Q = np.zeros((num_rows, num_cols, num_actions))
+
+# Define helper function to get possible actions in a state
+def possible_actions(state):
+ row, col = state
+ possible_actions = []
+ for i, action in enumerate(actions):
+ if action == 'UP' and row > 0 and grid_world[row - 1, col] != 'W':
+ possible_actions.append(i)
+ elif action == 'DOWN' and row < num_rows - 1 and grid_world[row + 1, col] != 'W':
+ possible_actions.append(i)
+ elif action == 'LEFT' and col > 0 and grid_world[row, col - 1] != 'W':
+ possible_actions.append(i)
+ elif action == 'RIGHT' and col < num_cols - 1 and grid_world[row, col + 1] != 'W':
+ possible_actions.append(i)
+ return possible_actions
+
+# Q-learning algorithm
+num_episodes = 1000
+for episode in range(num_episodes):
+ # Initialize the starting state
+ state = (0, 0) # start state
+ while True:
+ # Choose an action using epsilon-greedy policy
+ if np.random.uniform(0, 1) < epsilon:
+ action = np.random.choice(possible_actions(state))
+ else:
+ action = np.argmax(Q[state[0], state[1]])
+
+ # Perform the action and observe the next state and reward
+ if actions[action] == 'UP':
+ next_state = (state[0] - 1, state[1])
+ elif actions[action] == 'DOWN':
+ next_state = (state[0] + 1, state[1])
+ elif actions[action] == 'LEFT':
+ next_state = (state[0], state[1] - 1)
+ elif actions[action] == 'RIGHT':
+ next_state = (state[0], state[1] + 1)
+
+ # Get the reward
+ if grid_world[next_state] == 'G':
+ reward = 1 # goal state
+ elif grid_world[next_state] == 'H':
+ reward = -1 # hole
+ else:
+ reward = 0
+
+ # Update Q-value using the Bellman equation
+ best_next_action = np.argmax(Q[next_state[0], next_state[1]])
+ Q[state[0], state[1], action] += alpha * (
+ reward + gamma * Q[next_state[0], next_state[1], best_next_action] - Q[state[0], state[1], action])
+
+ # Move to the next state
+ state = next_state
+
+ # Check if the episode is terminated
+ if grid_world[state] in ['G', 'H']:
+ break
+
+# Print the learned Q-values
+print("Learned Q-values:")
+for i in range(num_rows):
+ for j in range(num_cols):
+ print(f"State ({i}, {j}):", Q[i, j])
+```
+
+## Conclusion
+Congratulations on completing your journey through this comprehensive guide to reinforcement learning! Armed with this knowledge, you are well-equipped to dive deeper into the exciting world of RL, whether it's for gaming, robotics, finance, healthcare, or any other domain. Keep exploring, experimenting, and learning, and remember, the only limit to what you can achieve with reinforcement learning is your imagination.
diff --git a/contrib/machine-learning/sklearn-introduction.md b/contrib/machine-learning/sklearn-introduction.md
new file mode 100644
index 00000000..7bb5aa8d
--- /dev/null
+++ b/contrib/machine-learning/sklearn-introduction.md
@@ -0,0 +1,144 @@
+# scikit-learn (sklearn) Python Library
+
+## Overview
+
+scikit-learn, also known as sklearn, is a popular open-source Python library that provides simple and efficient tools for data mining and data analysis. It is built on NumPy, SciPy, and matplotlib. The library is designed to interoperate with the Python numerical and scientific libraries.
+
+## Key Features
+
+- **Classification**: Identifying which category an object belongs to. Example algorithms include SVM, nearest neighbors, random forest.
+- **Regression**: Predicting a continuous-valued attribute associated with an object. Example algorithms include support vector regression (SVR), ridge regression, Lasso.
+- **Clustering**: Automatic grouping of similar objects into sets. Example algorithms include k-means, spectral clustering, mean-shift.
+- **Dimensionality Reduction**: Reducing the number of random variables to consider. Example algorithms include PCA, feature selection, non-negative matrix factorization.
+- **Model Selection**: Comparing, validating, and choosing parameters and models. Example methods include grid search, cross-validation, metrics.
+- **Preprocessing**: Feature extraction and normalization.
+
+## When to Use scikit-learn
+
+- **Use scikit-learn if**:
+ - You are working on machine learning tasks such as classification, regression, clustering, dimensionality reduction, model selection, and preprocessing.
+ - You need an easy-to-use, well-documented library.
+ - You require tools that are compatible with NumPy and SciPy.
+
+- **Do not use scikit-learn if**:
+ - You need to perform deep learning tasks. In such cases, consider using TensorFlow or PyTorch.
+ - You need out-of-the-box support for large-scale data. scikit-learn is designed to work with in-memory data, so for very large datasets, you might want to consider libraries like Dask-ML.
+
+## Installation
+
+You can install scikit-learn using pip:
+
+```bash
+pip install scikit-learn
+```
+
+Or via conda:
+
+```bash
+conda install scikit-learn
+```
+
+## Basic Usage with Code Snippets
+
+### Importing the Library
+
+```python
+import numpy as np
+from sklearn.model_selection import train_test_split
+from sklearn.preprocessing import StandardScaler
+from sklearn.linear_model import LogisticRegression
+from sklearn.metrics import accuracy_score
+```
+
+### Loading Data
+
+For illustration, let's create a simple synthetic dataset:
+
+```python
+from sklearn.datasets import make_classification
+
+X, y = make_classification(n_samples=1000, n_features=20, n_classes=2, random_state=42)
+```
+
+### Splitting Data
+
+Split the dataset into training and testing sets:
+
+```python
+X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.3, random_state=42)
+```
+
+### Preprocessing
+
+Standardizing the features:
+
+```python
+scaler = StandardScaler()
+X_train = scaler.fit_transform(X_train)
+X_test = scaler.transform(X_test)
+```
+
+### Training a Model
+
+Train a Logistic Regression model:
+
+```python
+model = LogisticRegression()
+model.fit(X_train, y_train)
+```
+
+### Making Predictions
+
+Make predictions on the test set:
+
+```python
+y_pred = model.predict(X_test)
+```
+
+### Evaluating the Model
+
+Evaluate the accuracy of the model:
+
+```python
+accuracy = accuracy_score(y_test, y_pred)
+print(f"Accuracy: {accuracy * 100:.2f}%")
+```
+
+### Putting it All Together
+
+Here is a complete example from data loading to model evaluation:
+
+```python
+import numpy as np
+from sklearn.datasets import make_classification
+from sklearn.model_selection import train_test_split
+from sklearn.preprocessing import StandardScaler
+from sklearn.linear_model import LogisticRegression
+from sklearn.metrics import accuracy_score
+
+# Load data
+X, y = make_classification(n_samples=1000, n_features=20, n_classes=2, random_state=42)
+
+# Split data
+X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.3, random_state=42)
+
+# Preprocess data
+scaler = StandardScaler()
+X_train = scaler.fit_transform(X_train)
+X_test = scaler.transform(X_test)
+
+# Train model
+model = LogisticRegression()
+model.fit(X_train, y_train)
+
+# Make predictions
+y_pred = model.predict(X_test)
+
+# Evaluate model
+accuracy = accuracy_score(y_test, y_pred)
+print(f"Accuracy: {accuracy * 100:.2f}%")
+```
+
+## Conclusion
+
+scikit-learn is a powerful and versatile library that can be used for a wide range of machine learning tasks. It is particularly well-suited for beginners due to its easy-to-use interface and extensive documentation. Whether you are working on a simple classification task or a more complex clustering problem, scikit-learn provides the tools you need to build and evaluate your models effectively.
diff --git a/contrib/machine-learning/tensorFlow.md b/contrib/machine-learning/tensorflow.md
similarity index 99%
rename from contrib/machine-learning/tensorFlow.md
rename to contrib/machine-learning/tensorflow.md
index 1d9357d0..b2c847c6 100644
--- a/contrib/machine-learning/tensorFlow.md
+++ b/contrib/machine-learning/tensorflow.md
@@ -61,4 +61,4 @@ TensorFlow is a great choice if you:
## Example Use Cases
- Building and deploying complex neural networks for image recognition, natural language processing, or recommendation systems.
-- Developing models that need to be run on mobile or embedded devices.
\ No newline at end of file
+- Developing models that need to be run on mobile or embedded devices.
diff --git a/contrib/machine-learning/transformers.md b/contrib/machine-learning/transformers.md
new file mode 100644
index 00000000..5a276885
--- /dev/null
+++ b/contrib/machine-learning/transformers.md
@@ -0,0 +1,443 @@
+# Transformers
+## Introduction
+A transformer is a deep learning architecture developed by Google and based on the multi-head attention mechanism. It is based on the softmax-based attention
+mechanism. Before transformers, predecessors of attention mechanism were added to gated recurrent neural networks, such as LSTMs and gated recurrent units (GRUs), which processed datasets sequentially. Dependency on previous token computations prevented them from being able to parallelize the attention mechanism.
+
+Transformers are a revolutionary approach to natural language processing (NLP). Unlike older models, they excel at understanding long-range connections between words. This "attention" mechanism lets them grasp the context of a sentence, making them powerful for tasks like machine translation, text summarization, and question answering. Introduced in 2017, transformers are now the backbone of many large language models, including tools you might use every day. Their ability to handle complex relationships in language is fueling advancements in AI across various fields.
+
+## Model Architecture
+
+
+
+Source: [Attention Is All You Need](https://arxiv.org/pdf/1706.03762)
+
+
+### Encoder
+The encoder is composed of a stack of identical layers. Each layer has two sub-layers. The first is a multi-head self-attention mechanism, and the second is a simple, positionwise fully connected feed-forward network. Each encoder consists of two major components: a self-attention mechanism and a feed-forward neural network. The self-attention mechanism accepts input encodings from the previous encoder and weights their relevance to each other to generate output encodings. The feed-forward neural network further processes each output encoding individually. These output encodings are then passed to the next encoder as its input, as well as to the decoders.
+
+### Decoder
+The decoder is also composed of a stack of identical layers. In addition to the two sub-layers in each encoder layer, the decoder inserts a third sub-layer, which performs multi-head attention over the output of the encoder stack. The decoder functions in a similar fashion to the encoder, but an additional attention mechanism is inserted which instead draws relevant information from the encodings generated by the encoders. This mechanism can also be called the encoder-decoder attention.
+
+### Attention
+#### Scaled Dot-Product Attention
+The input consists of queries and keys of dimension $d_k$ , and values of dimension $d_v$. We compute the dot products of the query with all keys, divide each by $\sqrt {d_k}$ , and apply a softmax function to obtain the weights on the values.
+
+$$Attention(Q, K, V) = softmax(\dfrac{QK^T}{\sqrt{d_k}}) \times V$$
+
+#### Multi-Head Attention
+Instead of performing a single attention function with $d_{model}$-dimensional keys, values and queries, it is beneficial to linearly project the queries, keys and values h times with different, learned linear projections to $d_k$ , $d_k$ and $d_v$ dimensions, respectively.
+
+Multi-head attention allows the model to jointly attend to information from different representation
+subspaces at different positions. With a single attention head, averaging inhibits this.
+
+$$MultiHead(Q, K, V) = Concat(head_1, _{...}, head_h) \times W^O$$
+
+where,
+
+$$head_i = Attention(QW_i^Q, KW_i^K, VW_i^V)$$
+
+where the projections are parameter matrices.
+
+#### Masked Attention
+It may be necessary to cut out attention links between some word-pairs. For example, the decoder for token position
+$t$ should not have access to token position $t+1$.
+
+$$MaskedAttention(Q, K, V) = softmax(M + \dfrac{QK^T}{\sqrt{d_k}}) \times V$$
+
+### Feed-Forward Network
+Each of the layers in the encoder and decoder contains a fully connected feed-forward network, which is applied to each position separately and identically. This
+consists of two linear transformations with a ReLU activation in between.
+
+$$FFN(x) = max(0, xW_1 + b_1)W_2 + b_2$$
+
+### Positional Encoding
+A positional encoding is a fixed-size vector representation that encapsulates the relative positions of tokens within a target sequence: it provides the transformer model with information about where the words are in the input sequence.
+
+The sine and cosine functions of different frequencies:
+
+$$PE(pos,2i) = \sin({\dfrac{pos}{10000^{\dfrac{2i}{d_{model}}}}})$$
+
+$$PE(pos,2i) = \cos({\dfrac{pos}{10000^{\dfrac{2i}{d_{model}}}}})$$
+
+## Implementation
+### Theory
+Text is converted to numerical representations called tokens, and each token is converted into a vector via looking up from a word embedding table.
+At each layer, each token is then contextualized within the scope of the context window with other tokens via a parallel multi-head attention mechanism
+allowing the signal for key tokens to be amplified and less important tokens to be diminished.
+
+The transformer uses an encoder-decoder architecture. The encoder extracts features from an input sentence, and the decoder uses the features to produce an output sentence. Some architectures use full encoders and decoders, autoregressive encoders and decoders, or combination of both. This depends on the usage and context of the input.
+
+### Tensorflow
+TensorFlow is a free and open-source software library for machine learning and artificial intelligence. It can be used across a range of tasks but has a particular focus on training and inference of deep neural networks. It was developed by the Google Brain team for Google's internal use in research and production.
+
+Tensorflow provides the transformer encoder and decoder block that can be implemented by the specification of the user. Although, the transformer is not provided as a standalone to be imported and executed, the user has to create the model first. They also have a tutorial on how to implement the transformer from scratch for machine translation and can be found [here](https://www.tensorflow.org/text/tutorials/transformer).
+
+More information on [encoder](https://www.tensorflow.org/api_docs/python/tfm/nlp/layers/TransformerEncoderBlock) and [decoder](https://www.tensorflow.org/api_docs/python/tfm/nlp/layers/TransformerDecoderBlock) block mentioned in the code.
+
+Imports:
+```python
+import tensorflow as tf
+import tensorflow_models as tfm
+```
+
+Adding word embeddings and positional encoding:
+```python
+class PositionalEmbedding(tf.keras.layers.Layer):
+ def __init__(self, vocab_size, d_model):
+ super().__init__()
+ self.d_model = d_model
+ self.embedding = tf.keras.layers.Embedding(vocab_size, d_model, mask_zero=True)
+ self.pos_encoding = tfm.nlp.layers.RelativePositionEmbedding(hidden_size=d_model)
+
+ def compute_mask(self, *args, **kwargs):
+ return self.embedding.compute_mask(*args, **kwargs)
+
+ def call(self, x):
+ length = tf.shape(x)[1]
+ x = self.embedding(x)
+ x = x + self.pos_encoding[tf.newaxis, :length, :]
+ return x
+```
+
+Creating the encoder for the transformer:
+```python
+class Encoder(tf.keras.layers.Layer):
+ def __init__(self, num_layers, d_model, num_heads,
+ dff, vocab_size, dropout_rate=0.1):
+ super().__init__()
+
+ self.d_model = d_model
+ self.num_layers = num_layers
+
+ self.pos_embedding = PositionalEmbedding(
+ vocab_size=vocab_size, d_model=d_model)
+
+ self.enc_layers = [
+ tfm.nlp.layers.TransformerEncoderBlock(output_last_dim=d_model,
+ num_attention_heads=num_heads,
+ inner_dim=dff,
+ inner_activation="relu",
+ inner_dropout=dropout_rate)
+ for _ in range(num_layers)]
+ self.dropout = tf.keras.layers.Dropout(dropout_rate)
+
+ def call(self, x):
+ x = self.pos_embedding(x, length=2048)
+ x = self.dropout(x)
+
+ for i in range(self.num_layers):
+ x = self.enc_layers[i](x)
+
+ return x
+```
+
+Creating the decoder for the transformer:
+```python
+class Decoder(tf.keras.layers.Layer):
+ def __init__(self, num_layers, d_model, num_heads, dff, vocab_size,
+ dropout_rate=0.1):
+ super(Decoder, self).__init__()
+
+ self.d_model = d_model
+ self.num_layers = num_layers
+
+ self.pos_embedding = PositionalEmbedding(vocab_size=vocab_size,
+ d_model=d_model)
+ self.dropout = tf.keras.layers.Dropout(dropout_rate)
+ self.dec_layers = [
+ tfm.nlp.layers.TransformerDecoderBlock(num_attention_heads=num_heads,
+ intermediate_size=dff,
+ intermediate_activation="relu",
+ dropout_rate=dropout_rate)
+ for _ in range(num_layers)]
+
+ def call(self, x, context):
+ x = self.pos_embedding(x)
+ x = self.dropout(x)
+
+ for i in range(self.num_layers):
+ x = self.dec_layers[i](x, context)
+
+ return x
+```
+
+Combining the encoder and decoder to create the transformer:
+```python
+class Transformer(tf.keras.Model):
+ def __init__(self, num_layers, d_model, num_heads, dff,
+ input_vocab_size, target_vocab_size, dropout_rate=0.1):
+ super().__init__()
+ self.encoder = Encoder(num_layers=num_layers, d_model=d_model,
+ num_heads=num_heads, dff=dff,
+ vocab_size=input_vocab_size,
+ dropout_rate=dropout_rate)
+
+ self.decoder = Decoder(num_layers=num_layers, d_model=d_model,
+ num_heads=num_heads, dff=dff,
+ vocab_size=target_vocab_size,
+ dropout_rate=dropout_rate)
+
+ self.final_layer = tf.keras.layers.Dense(target_vocab_size)
+
+ def call(self, inputs):
+ context, x = inputs
+
+ context = self.encoder(context)
+ x = self.decoder(x, context)
+ logits = self.final_layer(x)
+
+ return logits
+```
+
+Model initialization that be used for training and inference:
+```python
+transformer = Transformer(
+ num_layers=num_layers,
+ d_model=d_model,
+ num_heads=num_heads,
+ dff=dff,
+ input_vocab_size=64,
+ target_vocab_size=64,
+ dropout_rate=dropout_rate
+)
+```
+
+Sample:
+```python
+src = tf.random.uniform((64, 40))
+tgt = tf.random.uniform((64, 50))
+
+output = transformer((src, tgt))
+```
+
+O/P:
+```
+
+